Actual voting by slaves had nothing to do with it, though it should have.
It was completely for the purposes of representation in Congress.
If each slave counted as a full person that would give more representation, and thus larger clout when voting on slavery-related issues for the slave-holding states.
If slaves counted as zero in the census, there would have no increase in Congress for slave-holding states, and less relative clout.
But the assumption was that the relative size of the general population of a state roughly correlated with the relative size of the subset of the population that were qualified voters.
The subset of slaves who could vote was zero.