For the purposes of apportionment but not for voting.
Moving the goalposts.
Women couldn't vote back then, and neither could children, but both were counted for the purpose of apportionment. For that matter, non land owning white men couldn't vote either.
In those days, you had to own land to be able to vote.