The difference is the Lee family had no interest in owning slaves and freed all the ones they inherited once the debts were paid off. The Grants, however, had no such scruples and didn’t free theirs until they were forced to by the 13th amendment, or else their slaves ran away before that as you say they might have done.
That is simply a lie.
The Grants didn't live anywhere that slavery was legal after January 1865 so how could they have owned slaves until December 1865?