By your reasoning, Dumbledore could have been stated to be a pedophile, a rapist, or into bestiality also, and it would have made no difference to you.
Nice reasoned leap, from homosexual to pedophile. Unreasoned arguments cloud the issue. By my reasoning I can call John Galt a Communist and it won’t change the story that has already been written. There is no reference to the character’s sexuality in the books. Just because someone decides that the character is gay, even if it is the authoer, it does not change the story that has been written. Ultimately it is a work of fiction that has only the social impact that we give it.