In an essay I wrote called the Constitution and the Confederacy, I state less that one in a hundred Southerners could have been motivated by slavery. Winston Churchill in volume three of his History of the English Speaking People, begins discussion of the subject by saying, ”An aristocracy of planters living in rural magnificence and almost feudal state, and multitude of smallholders, grew cotton for the world by slave labour. Of the six million white inhabitants of the so-called ‘southern states’ less than three hundred and fifty thousand owned slaves, and only forty thousand controlled plantations requiring a working unit of more than twenty field hands. But the three or four thousand principal slave-owners generally ruled the politics of the South as effectively as the medieval baronage had ruled England.”
That's a misleading statistic. Almost all of those 350,000 slave owners had spouses, so that means up to 700,000 people directly benefited from slavery. Say each of those families had three children and suddenly you are up to 1.8 million people people drew direct benefit from slavery. That's large enough percentage of your population right there to understand why they may rebel to protect their slave property.
Aggrieved by the action and tendencies of the Federal Government, and apprehending worse in the future, a majority of the people of the South approved secession as the only remedy suggested by their leaders. So travelers enter railway carriages, and are dragged up grades and through tunnels with utter loss of volition, the motive power, generated by fierce heat, being far in advance and beyond their control.
Destruction and Reconstruction is a great read, plus I'd like to recommend for further reading, anything you can find on "The Red River Campaign".