In the last years of slavery in the U.S., only 10% were free
After Nat Turner's Rebellion, slave slaves began to restrict manumission of slaves, not expand it. There is no real evidence that the South was moving towards freeing the third of its population that were enslaved.
The influence of Free Blacks was growing, not just those coastal Free Men of Color who owned slaves.