And what, please tell me, did the institution of slavery affect the most? The economy.
Absolutely. Keeping slaves was an economic decision. The only reason to own a slave is to have the use of slave labor.
The loss of slaves caused economic devastation to the South. A good slave was worth $1000, an incredible sum back then.