I think your reasoning for his defense is exactly how he gets away with it.
Predators are often those with the best possible reputations. They rely on these good reputations to help them get away with it and plant doubt in the direction of the accuser (who is often a real victim).
How many people didn’t believe that a Catholic priest would harm just one kid, much less dozens?
I agree. Besides clergy (not only priests) read about the UK star-maker Jimmy Savile at the link above.
This link shows how the British MSM (BBC) buried the Jimmy Savile story after they knew.