When I was growing up my mother didn't drive. We had one car and it was titled in my father's name. You could truthfully say that only 20 percent of the family owned a car. But 100% got benefit from that car ownership. The same is true with slave ownership. It may be true that only 5% of all Southerners owned slaves. But if all of them were married then suddenly 10% of all Southerners derive benefit from that slave ownership. Add a couple of kids and the percentage goes up. The more accurate comparison would be to look at the number of slave owners and the compare it with the number of families listed in the 1860 census. Look at that and you see that in states like Mississippi half of all families owned slaves. In the confederacy as a whole between one-third and one-quarter of all families had slaves. Look at it in those terms and it's easy to see why they would fight for it.
...and lived in worse economic conditions than many slaves.
Nonsense.
They had no vested interest in slavery, but they wanted to keep their Southern culture
A culture built on slavery.
The claim that "only about 5% of whites owned slaves in the South," is true, in a sense, but also a canard.
According to the 1860 census, there were about 8 million whites in slave owning states. Of those, 385,000 owned slaves -- or about 5%. But these were all relatively wealthy heads-of-households, meaning they also had large families.
If the average immediate family size was, say, six (a wife and four children), now we see that about 30% of southern whites lived in the homes of slave holding families.
And consider a typical young white farming family in the Deep South. If they did not themselves own slaves, their parents did, and the young family would too -- as soon as they could afford them. So they were in no sense "anti-slave."
And that is just "average" for the entire South, where the slave population ranged from well over 50% in Deep South states like South Carolina and Mississippi, to about 25% in Upper South states like Virginia and Tennessee, to barely 10% in Border States like Maryland and Missouri.
So, while the average of slave-owning white families may have been 30% overall, the range was from well over 50% in the Deep South to under 10% in Border States.
Indeed, in states like (western) Virginia, (eastern) Tennessee and (western) North Carolina, many counties of the state had relatively few slave owning households and so refused to seceed, or resisted secession. Some of these also supplied soldiers for the Union army.