See the 23rd Amendment.
That amendment only designates who can represent the district, but does not define the district or Congress' powers to change that district. In fact, it muddies the water further by stating: "The Congress shall have power to enforce this article by appropriate legislation."