And that time wasn't too long ago. Ever watch an old World Series or All Star game on the ESPN Classic Sports channel? Up until the early/mid-90s most players were not much larger than the average guy.
When I was a kid I used to collect the Fleer old-time baseball cards, and it was amazing to me that the players of the 20s and 30s often looked prematurely aged (i.e., in their late 40s or 50s) even during their careers. I think in those days the average person didn't have all the early health care, vitamins, shots, etc., that we now take for granted.