I once saw a person interviewed on a talk show who had done a female to male sex change. In their clothed appearance and demeanor, you would have thought this person was unquestionably an extremely masculine man. The radio show This American Life also featured an interview with a lesbian who had undergone the same process. The interesting part was the psychological changes she/he described as occurring after beginning to take testosterone injections. These included violent sexual urges, a male-like focus on the physical attributes of women, and an increased interest in science.
The factor that both cases seemed to point to was the very powerful effect of testosterone on the mind of persons who had not previously been exposed to it.