Grant believed the south had a right to secede. I think he is fairly clear here:
The fact is the constitution did not apply to any such contingency as the one existing from 1861 to 1865. Its framers never dreamed of such a contingency occurring. If they had foreseen it, the probabilities are they would have sanctioned the right of a State or States to withdraw rather than that there should be war between brothers.
In a letter to Elihu Washburn dated Aug 30, 1863. “The people of the North need not quarrel over the institution of slaver. What Vice President Stevens acknowledged the cornerstone of the Confederacy is already knocked out. Slavery is already dead, and cannot be resurrected. It would take a standing army to maintain slavery in the South, if we were to make peace today, guaranteeing all their former Constitutional privileges. I was as never an abolitionist, not even what could be called anti slavery; but I try to judge fairly and honestly; and it became patent to my mind, early in the rebellion, that the North and South could never live at peace with each other except as one nation, and that without slavery. As anxious as I am to see peace established, I would not, therefore, be willing to see any settlement until this question is forever settled.
A few years later communicating with Otto Von Bismarck “but as soon as slavery fired upon the flag, we all felt even those who did not object to slavery, that slavery must be destroyed. We felt that it was a stain to the Union that men should be bought and sold like cattle.”