I suppose that is possible but that should not be a reason for not doing what is right.
Also, if he was exposed, the same thing would have happened. It just can't be stopped.
The circles I run in, nobody would be gossiping trying to figure out who it was. Mostly the attitude is that it's nobody's business and it's irrelevant to know who was involved.
He sinned against his wife not them. Why do this to her? Really the best thing was for him to announce that he was stepping down and let someone else take over the church while he worked on his marriage. He need not tell anyone why. Trust me I was a protestant, once. Nothing can make a church uneasy than to know the preacher man had an affair. Even worse than to deal with the preacher's wife after all that. Talk about walking on eggshells. That is drama I seriously don't need and makes me so glad I no longer have to deal with it.