The correct response to such people is "OK. So?"
The underlying assumption is that people are under a moral obligation of avoid being transphobic/islamophobic/sexist/racist/whatever. My response is "I reject your assumptions on what I need to do and how I need to feel".
If I feel like only dating blonde Swedish women with natural-born vaginas and XX chromosomes, then that is exclusively my business and I shall feel no obligation to explain or justify my preferences.