And General Grant's wife's slaves were not released until after the Civil War.
Incorrect. According to Missouri archives the Dent family slaves, including the ones Mrs. Grant had use of, were freed shortly after the Emancipation Proclamation was enacted and there are no reports of Mrs. Grant being seen with them after that. In any case, Missouri amended her state constitution to outlaw slavery in January 1865 so Mrs. Grant could not have legally held any slaves after the rebellion was over.