Obviously gestation is very complex and has different stages. The connection between the development of the genitals and parts of the brain is certainly not fully understood. Studies of finger-length ratios suggest that male homosexuals might even have a higher-than-normal level of testosterone in utero. I would assume that with intersex cases the problem is often the opposite.
It is well established that a group of women whose mothers took a particular drug to avoid miscarriage grew up to be unusually promiscuous. I have met one of these women (dated her sister). So clearly the pre-natal environment can affect one's adult sexual nature.
Dont assume. Ambiguous external genitalia develops in females when the external genital structures are exposed to greater-than-normal amounts of male hormones resulting in masculinized females. There are several syndromes associated with this problem, mosaic comes to mind.