Seriously, I'm 6'3" 250 pounds and have completed Ironman triathlons. When I was in my 20's I could run a sub-eleven second 100. I doubt many women can say the same.
I don't know what that proves, other than that you are/were in great shape.
My fiancé is a small man, a career officer, who has always weighed about 10st (less than 140lbs). He is retired from one of the British Special Service Regts. You know what those are, right? Do you know what their training consists of?
I can't discuss what he is doing now, only that he is with the British army on the front in Afghanistan. Most women I know weigh more than he does, and many are taller. Size isn't everything.