Perhaps you can demonstrate where "the people" who voted were not adult, white, male citizens? Perhaps, as you say, one or more states may have allowed women to vote? Or children? The insane? Non-citizens? Slaves, perhaps?
Or are you saying that "the people" of Article I, Section 2 are different than "the people" of the second amendment?
Sure. Long before the 19th Amendment was passed, several states granted the franchise to women; for instance, New Jersey (which granted women the franchise at the time of the adoption of the constitution), Wyoming, and Illinois. By the time of the adoption of the constitution, black men were allowed to vote in most every northern state, and in some southern states, including Maryland, North Carolina, and Kentucky.
As I said before, "the People" encompasses everyone, although some states may have--properly--drawn narrower limitations.