No, given to them by the State of South Carolina in 1836: "Resolved, That this state do cede to the United States, all the right, title and claim of South Carolina to the site of Fort Sumter and the requisite quantity of adjacent territory"
If given by the consent of the people, can be taken by the consent of the people.
In retrospect, it would appear that the smarter move would have been to leave them alone and let them keep it until they got bored with it, much as Cuba has been doing with Guantanamo since the Revolution.