There are still some youngsters out there, but it seems that most technically inclined kids are attracted to computers and the net rather than radio these days.
As for "old men", I guess that depends upon your definition of old. I was first licensed at 22 in 1983 and just turned 45. When I was in high school that would have almost qualified as ancient, but the mid-40s don't seem quite so crusty from my present perspective. ;-)
A couple months ago I "talked" (via Morse code) with an 87 year old gent who had been active in the hobby since 1930. I have also had QSOs (conversations) with hams as young as 12. The trend is definitely towards the former rather than the latter, though, just as in many other hobbies. If I had to guesstimate an average age based upon the hams I know, I'd put it somewhere around 50.
I was licensed in 1979 and graduated from high school in 1983. I just joined QCWA for the fun of it. But I don't consider myself old yet either!