Because your professional responsibilities seldom make you crazy, and rarely invoke that part of your brain that is involved in having an affair.
There is no "affair" part of the brain - junk science doesn't excuse this kind of behavior.
If a man cannot honor his solemn vows, if he can't respect his own family, if he has no moral integrity, then he cannot be counted on.
Period. End of story.