I'd say this is an over generalization and in many places, not accurate. Some areas that were especially conducive to the production of cash crops had large plantations. Others did not. Of the 5.63% of White Southerners who owned slaves, half of them owned fewer than 5. So the whole Planter class of large slaveowners and large plantations was less than 3% of the population. Did they have an outsized influence? Sure. The rich always do. Did they totally dominate? No. They could and sometimes were outvoted by middle class yeoman farmers. The farms they owned were by and large not impermanent. They were family holdings they owned outright and passed down through their families.
Several close friends were of such stock. And, as with others of that background, they provided the Confederacy with fighting men. As the saying went, it was a rich man's war, but a poor man's fight. Three friends had ancestors who were Confederate officers and small scale farmers and slaveholders.
Another family friend had ancestors who were significant plantation owners and political figures in Tallahassee. One such plantations became the City of Tallahassee Golf Course and Country Club, complete with unmarked graves of slaves and a dodgy maneuver that kept blacks out as members for decades.
Yet another friend in Tallahassee was a small businessman with ancestors who arrived in the 1830s of the hardscrabble type. His ancestors never owned slaves. Good hearted and tough as an old tree root, my friend clashed with the powers in local politics. Years after his death, many of his adversaries were prosecuted for corruption.
My excuse for bringing up old friends who have passed is that, as Faulkner put it, "The past is never dead. It's not even past."
I had 9 direct ancestors serve in the Confederate army including my G-G-Grandfather who was a captain (officers up to the rank of captain were elected). He had been a 23 year old Sheriff of his county before the war. His 5 brothers also served. His youngest brother was murdered during the war by bandits when he was bringing some of the family's horses from one location to another. One of his brothers was killed in the war and 2 were seriously wounded. 2 of my other ancestors died in the war - one of measles in camp Trousdale (the main training camp in Tennessee near present day Nashville) when he was an 18 year old kid fresh off the farm and his 23 year old brother in the battle of Chickamauga.
Of all of my ancestors' families who lived in Tennessee at the time. there were as of the 1860 US Census, a grand total of 0 slaves owned. This was typical of the vast majority of the population. By the way, my direct name bearing line arrived in Jamestown in 1649 so it wasn't like they were recent arrivals. Another of those families were recent arrivals from England and another had ancestors who served in the North Carolina Militia during the War of Independence and fought in the Battle of Guilford Courthouse (I have an account he gave and somebody else wrote down many years after the war) - so they were here quite a while themselves.
Amazingly, US census records are good enough that I was able to trace a black friend's ancestry back to two specific slaves on an Alabama plantation, then to a series of small farms after the Civil War. She was delighted. Since almost every black person in the US is descended from slaves, discussion of the subject of slavery cuts more deeply for them.
My grandparent were all immigrants early in the last century with no connection to the US Civil War. The closest that any recent relative came to combat was an Irish grandfather who seems to have been involved in the Irish Civil War, a great uncle who died while in the British Army on the Somme in WW I, and an uncle who was in the US Merchant Marine on the North Atlantic run during WW II. My father was in the US Navy during the Korean War. I also had a great uncle who was in Rural Solidarity in Poland and was apparently murdered by the communist secret police.