Well, remember, all those who served the Union were promised their freedom in return, and it seems to me, iirc, also those very few at the war's end who were finally enlisted in the Confederate Army.
Indeed, I'm certain that's one reason the Confederate leadership delayed enlisting blacks until the very last bitter-end.
They didn't wish to promise freedom for slaves.
jmacusa: "A slave revolt in a victorious south would have been a horrifying situation, most especially for the slave population."
Obviously a victorious Confederacy would have a much easier time of it than a defeated one.