The problem happens when physical and mental development happens in the developing body in a way that defies the sexual chromosome configuration a person has. This is a reason why there are some very feminine women - who never identified or were even recognized as anything else, all growing up - that just happen to have "XY" chromosome pairs.
To those women - and they are women - I feel very sorry, as they can't have kids, and if the information about their chromosome configuration becomes public, can lead to ignorant bashing of themselves and their boyfriends / husbands / adopted children.
(Yes, this is a reply to a year-old thread. I was looking up Michael Phelps, and came across this thread....)