“Notice what the Constitution authorizes: Congress may accept cessions from states to form a District. It does not authorize Congress to return, retrocede, or give away territory once accepted”
Read ARTICLE IV., SECTION 3
“dispose of...the territory...”
Looking at the history, I believe your view is correct.
Well played.