Well, since she was one of his employees, I assume he knew her marital status.. However, even if he didn't know, it is still adultery. It may mitigate his culpability on that score, but he was still involved in a sexual relationship with another woman while he was still married. That constitutes an affair.
My dad died when I was rather young but he did tell me "Don't F@ck the help"