Did he confess to each and every one of the voters to whom he sold himself as an advocate of "family values"?
No, he did not. (Well, not until he knew he was about to be outed anyway, which doesn't count.)
A short term lapse is forgivable. If he had a history of repeated offenses’ that is a different story.
Have his votes been on favor of immoral positions? If not, he has not let a personal failing affect what he knows is right.
Have you always been this judgmental?