They had in mind the following: members of American Indian Tribes, while on tribal land, provided the Tribe had status as an independent, sovereign nation; persons in the US under diplomatic immunity. Those are the only persons within the country's borders who were not (at the time the Ammendment was written) legally obligated to subject themselves to Federal sovereignty over their persons. Such persons, if brought before a Federal magistrate, could rightfully argue that the court had no jurisdiction over them.
So prior to 1924 when Congress recognized American Indians as being included in the 14th Amendment what would have been the case if a woman who was a resident of a tribe went to a city hospital to give birth? Even though the hospital would have been within the boundaries of the US automatic citizenship was denied to her kid because she was not technically a citizen of the US. That's my understanding of how the law worked.
The same can be said of illegal aliens... they are not residents of this country and can be excluded unless Congress, as they did with American Indians chooses to include them.