That used to be my argument, but now I realize that the question is about the culture attracting homosexual men to it.
An institution of powerful robe wearing men, all vowing to never know a woman, to never be husbands, boyfriends, or fathers, to devote themselves to the world's most exclusively, global, female excluding all male club. Something like that is bound to attract a lot more than just normal men who agree to celibacy.
Following Christ involves sacrifice - "Take up thy cross, and follow me." It is precisely the fact that a man who answers God's call to the priesthood makes a tremendous sacrifice. In return he receives tremendous graces.
And of course this is exactly why homosexuals should not be allowed in the priesthood: they aren't giving up anything. No concept of sacrifice - for them it IS just an old boys club. And that's part of the problem.
. . . and you must never have been inside a Catholic parish, there are females all over the place. Just not with collars on.