I'd imagine the years of carpentry would have built up his arms and shoulders more than the "swimmer's body" he's sometimes portrayed with.
I agree - that's part of my reasoning that he was larger than the average man of that day.