I think it all goes back to his treatment at that clinic when they cured him of his attraction to women. Now that he never looks at or thinks about women, he has nothing to look forward to after a long day.
I doubt that anything can be done, though, because the cure is usually permanent and irreversible.