Actually, even if only one is married, it is still extramarital.
So maybe it’s extraextramaritial or extramaritialmariatial or extramaritialextramaritial.
It sounded like it was his fault only.