At the time of some of the decisions the crackpot birthers cite, Congress had NOT entered the picture on the issue, directly.
Once Congress did act, Congress became the “elsewhere” -—
And James Madison, Father of the Constitution, clearly stated that Congress had the authority to act on Citizenship matters.
According to Article 1, Section 8, para 4, Congress is authorized to establish uniforn rules of "naturalization." And they have done so. Foreigners and aliens who are not natural-born become citizens through the processes Congress has defined.
The Court in Minor established the meaning of "natural-born", setting precedent which Congress is not free to change. Here are the options (as explained by Donofrio) open for changing that understanding.