Does anybody really question that most people can make a pretty good judgement about whether some people are gay or not based on their voice or mannerism or appearances?
The question, though, is just how far that will take you. People who claim to have "gaydar" say or believe that they can come to conclusions with a lot more accuracy than one can seriously claim.
They aren't saying, "Hey, sometimes I can guess who's gay and who isn't." They're asserting that their sense is a lot more infallible than that, and they don't.
I'm not sure it's a liberal-conservative thing, since the greatest believers in "gaydar" are gays themselves (at least until they want to feel indignant about somebody else claiming to have it).
Mostly I hear it in a context like when Liberace announced that he was gay. It's more of a "no kidding" than a "I can detect hidden thoughts".