You might want to research that. "planter class" a well-known term in historiography and it has a meaning. If your ancestors were engaged in agriculture and weren't using slaves, they were farmers, not planters.
I've said on these threads before that slaveowners must have been the least fertile people on earth. No one seems to be descended from one.