The Constitution authorizes Congress "To exercise exclusive Legislation...[and] Authority over all Places purchased by the Consent of the Legislature of the State in which the Same shall be, for the Erection of Forts, Magazines, Arsenals, dock-Yards, and other needful Buildings" (emphasis added)
It does NOT authorize Congress to exercise similar authority over forts that are ceded or lent to the federal government. The founding fathers recognized this key distinction and, accordingly, passed a 1794 enabling act that would allow the states to conditionally cede forts for the purpose of garrisoning them with federal troops. South Carolina did this in 1805 with three existing forts in Charleston. In 1827 they used the same process, and not the federal purchasing process, to cede the land for Fort Sumter.
Had they said to Congress "we will sell you the land on which Fort Sumter will be built" and Congress responded by providing payments, then yes - that clause of the constitution would govern Congress' jurisdiction. But that is not what happened, hence the 1794 Act governs it.