It was treated as a privilege and not a right by our Founding Fathers. Not everyone could vote. It was reserved for those who had more at stake and would take therefore take it seriously.
Amendment XXVI
Section 1. The right of citizens of the United States, who are 18 years of age or older, to vote, shall not be denied or abridged by the United States or any state on account of age.