A man who commits adultery loses respect in my eyes. That being said, when I vote for President I am not voting for pastor, and with very few exceptions I have to hold my nose when I mark the ballot.
The way I see it, Jimmy Carter was faithful to Rosalyn, but he was still a horrible President.
So give me the philanderer who cuts taxes and is for smaller government, over the faithful candidate who is for raising taxes and bigger government.