The last slaves were freed in New Jersey.
Legally, whatever slaves remained in any state, North or South, for any reason except criminal conviction, were freed by ratification of the 13th Amendment in December 1865.
Whatever "servants" wished to remain with their previous "masters" was their choice, but the law no longer bound them, period.