That is sort of true and sort of not true.
People take that as that they can think about it and have fantasies and as long as they don't actually do it they are ok.
But thought is the father of action.
That is why Jesus taught about lusting in your heart.
If you harbor and cuddle your lust one day it will break free.
It is how humans work.
You did get right to the heart of the matter, the human heart. I agree with everything you wrote.