IOW, they were dowry slaves. The same situation as George Washington. And you weren’t allowed to free dowry slaves — or to free slaves if you were in debt.
When G.W.P. Custis (Mary’s father) died in 1958, he owed about 75 slaves. The majority of those slaves worked in and around the estate at Arlington, but some of his slaves worked on other Custis properties as far South as West Point, VA. His will required all the slaves to be freed when the “legacies and debts of the estate were paid in full”, or within five years of his death. Lee was the administrator of the Custis estate. The last of the Custis slaves were freed by Lee in Dec 1862.