In some states of the South, the slave population was almost as much as the White population. There was a true concern that if freed, there would be violent payback. Also, most of the North were slave states at one time, and abolished slavery gradually by passing laws (NY effective 1800) that any slave born after a certain date was free. That led to NY slaves being sold to owners in slave states. Virginia's legislature almost voted to abolish slavery a few years before the civil war. Slavery's end was inevitable, but it may have taken until 1890 or 1900 for slavery to end in the South (without the war).
Virginia's constitution did not allow the legislature to pass any acts of manumission. It also provided that any slave freed had 12 months in which to leave the state or be sold back into slavery.