Goodwill, love, and enthusiasm for the marriage is like a bank account. Anytime you do something nice, you add to the account. Everytime you cause dissappointment and pain, you take away from the account. Wives have no idea just how much they deplete that account by frustrating their husbands sexually. You can't simply change and make things better, you have to replace all the negative feelings he has with positive feelings. Realizing that when you are that far in the hole, the inevitable slipup will set you back far more than in the beginning.
Even though you spent five years changed, he undoubtedly still took a negative view of you and the marriage and it may be his revenge for the misery he suffered. Especially since his pain was great enough that he can only see your efforts in hindsight. It also depends on what "elsewhere" was, I am assuming another woman. The thing is, he had a very "easy" relationship with her. No kids, no mortgage, no stress, just the good stuff. If he is still with you then you are seeing some character because no wife can compete with that. Unless he has completely cut off contact with her, that is always going to be a strong temptation for him.
Of course, his action depleted his account with you, and now you don't want to put forth the effort either. You are where he is/was.
When you wound someone enough, they come to expect you to wound them and they won't be all that careful not to wound you. Eventually someone else comes on the scene and they have to make the choice between this new person who hasn't wounded them and the person who has spent years hurting them.
I do realize the hurt I caused him. I even placed much of the blame on myself. A year later though, I find myself so angry at times. The whole he wasn't perfect, either, thing...
And then there's all the baggage that goes along with a 17 year marriage. Hard to pack it away and begin again. But that's what we'll have to do if we're going to make it.
Ah, well...life is tough and then you get married. ; * ) I completely understand what you're saying, and if nothing else it has given me a few more days of strength and patience. Thanks...