In addition to the well-known brow ridge (which early Homo sapiens had and some people still show), Neanderthals had a low-crowned brain case extending in a "bun" in the back. They had bell-shaped rib cages which together with short lower vertebrae and wide hips meant they had almost no "waist" area. That is, their rib cages hung low over their hips, reducing flexibility but giving them better armor against the kind of "gut wound" that was inevitably fatal until sometime in the 20th century. They also had rather funny upper-arm/lower arm and upper-leg/lower leg ratios. (The lower parts of the limbs were short, outside the range of modern variation.)
You're right that they were "built tough." They show the same pattern of injuries as veteran rodeo cowboys: lots of healed breaks. In hunting big animals, they apparently didn't use the kind of weapons you throw or shoot from a distance.
I can't argue with this point but, I do wonder who was using these 400, 000 year Old spears found in Germany.