Absolutely. It hurts many, many people- not just the spouse/children. Of course they are the main ones effected, but the cheater is now untrustworthy in the eyes of many people in a way they usually do not anticipate.
Maybe if these pols’ wives would make it clear to their meal tickets...er, spouses, that adultery was a deal breaker, that might help their menfolk find some self-control, if not some decency.