I grew up in a large family, 8 kids. My dad worked and my mom didn't drive so while we had a car, it was registered and licensed in my father's name and he was the sole driver. So one way of looking at it is that only 10% of the people in my family owned a car. But we all reaped the benefits of car ownership. Likewise with slavery. The total number of slave owners may have been around 6 or 7 percent, depending on the state, but undoubtably almost all of those slave-owners had wives and children. Looking at it that way, almost 50% of the families in Mississippi owned slaves. Over 40% of the families in South Carolina and Georgia. Over all, almost 30% of the families in the south owned slaves. So that is why it was so important to them. They weren't fighting to defend the rights of 6% of the population, they were fighting to defend an institution that almost a third of them had directly benefited from.
No matter how you arrange the statistics, the South of the 1860's was agricultural. Practically every slave owner worked in the fields with his slave labor, while the children did the same.
This points out the absurdity of the Slave Aristocracy concept. However, you did assert the big fallacy that the South was fighting to preserve slavery. Remember, it did not have to fight, the slave states had seceded and specifically legalized slavery in these states.
The South fought to keep the armed people of Mass., RI, NY, Ohio, Pa., NH, IL., etc. off our land.
You should be asking why, if slavery was gone from the North due to secession, were these people were ordered to our soil?