What the Msgr leaves out is at this point the man has committed adultery. He's as guilty with this thought as he would be if he were to engage in the physical act itself.
That's taking it way too far. Adultery is the physical act of sex with a woman not your wife. We all have sexual thoughts, this is designed in by God.
Somebody toys briefly with the idea of stealing a pencil, while somebody else commits armed robbery of a bank and gets away with $200,000 in cash. Both of them are committing the sin of "stealing," but it's ridiculous to suggest that the objective guilt of both acts is the same.
Otherwise you end up with morally ridiculous situations, like equating what Adolph Eichmann did with me getting angry at someone who cuts me off in traffic. Both are sins against the commandment "Thou shalt not kill".