Bet you couldn't distinguish my skull from that of a neanderthal. I have bony flaps over my dental nerves, protruding eye ridges, an overly large skull, and too many teeth. ;)
Thankfully, I don't plan to do that. :)
I prefer my corpses to be at least 50,000 years old!
On a more current note, as you imply, external characteristics only give a small hint - no reputable physical anthropologist that I know, would make such an inference.
You really should post a pic! LOL