So he not gay. He is bi-sexual. That means that he has choice.
He chooses to have sex with men rather then honor his vows to his wife.
If having sex with men was so important to him THEN HE NEVER SHOULD HAVE MARRIED AND HAD KIDS.
There are true homosexuals who are only attracted to their own sex. Mr Ashburn is not one of them. He just wanted to excuse his catting around.