Lincoln's emancipation proclamation eventually freed millions in the Confederacy.
As for conscription: Confederates began drafting in April 1862, the Union in July 1863.
Cartoon from the time showing Confederates drafting a Southern Unionist:
The North had the opposite problem, the cause "freeing the black man" wasn't worth dying for and frankly to them it sucked, but the living conditions were much, much better for them.