With respect, Grant's wife and her family came from Missouri and Grant's father-in-law was a slave owner. Grant himself owned a slave, given to him by his father-in-law, for a brief period in 1858-59 but when he moved to Illinois Grant gave the man his freedom rather than sell him, even though Grant could have used the money. Mrs. Grant had the use of several of the Dent family slaves on and off during her married life but all of the Dent slaves were freed in January or February of 1863.
The records indicate that Grant personally owned a slave named William Jones between 1858-1859. His wife Julia also brought four slaves of her father with her when they married. In 1858 Grant is also known to have written to his father that "I have now three Negro men, two hired by the year and one of Mr. Dent's," referring to his personal slave. As for where the slaves were kept, it appears that he and his wife took them with them when they were in slave states and territories. When Grant went north on military duties, they were left in Missouri and Kentucky, taking care of the children.