Yeah. My ex was convinced that if I came home 5 minutes late on a 40 minute commute that I must be having a affair.
***
A friend of mine had a similar experience. Ex-husband always checking up on her....giving her heck for being late, and yes, actually accusing her of stepping out on him.
Turned out that it was HE who was having the affair.
My friend said that when your spouse or companion accuses you of having an affair, chances are he/she is the one who is.
OTOH, I have studiously avoided getting entangled ever since.