“If it had not been for Slavery, the South would have never left the Union in the first place.”
Then you need to study the taxation situation and other issues of the times. Slavery was but one of many economic and power issues of the day. To think that the average southerner that did not hold a single slave looked upon those fields with slave labor and not desire the job themselves is sort sighted.
Slavery is more than just an ‘economic issue’.