You provided no support for your argument at all.
“Southerners hated slavery.”
“Very few, less than 1.4% ever held slaves at all.”
{I looked it up, was up to 6% at the peak}
” Slaves displaced paid workers.”
You made 3 statements, and none of them have any relevance to the question of whether the South would have been better off or more economically developed or more advanced if it had never had slaves at all.
So why do you believe it?
These are personal statements, not research articles.