Not all of them. Washington was over 6' tall. Madison was the shortest President, but also the brainiest.
Shorter than Adams? ;-D