The Congress has no power beyond its borders to dispose or retain military structures in a foreign nation, should that nation chose to order the garrison to vacate. That is exactly what happened to provoke the War for Southern Independence. The Confederate States of America was its own nation; and had jurisdiction to remove hostile forces.
The why are we still in Guantanamo Bay? The Castro government has demanded for 40 years that we leave there. They've cut off water and power and all contact with the rest of the island all in an attempt to get us to pull out. So by your definition then we should have ordered the troops home back in the 1960's and turned the whole thing over to the commies. Right? We have to be in the wrong on this because the foreign nation has ordered us to leave. Failing that should Castro decide to bombard the base into surrender then you would believe he was in the right?