Are you missing the fact that this law dealt with Naturalization? Not with those born in the US?
Are you missing the fact that had it dealt with carriage wheels, it would still indicate the mind of the congress? As I have mentioned ad infinitum, an act of congress cannot repeal or modify an article of the constitution. It can only show what the framers thought about something. In that regard it is all too clear. They didn't care about place, they did care about Non-American fathers. Madison included.