And the illiterate too. I think the word you are looking for is "innumerate".
I wasn't looking for it. But now, the etymology of innumeracy I did look for but no dice yet...