I don't think the argument is whether or not priests are perfect. I think everyone knows they are not. No one expects them to be, but we certainly don't want them to fall below what good society regards as wholesome.
There is too much of a belief among many that enforced celibacy ensures some kind of higher spirituality. This is not true and other branches of christianity know it.
I would say most Catholics don't believe that, though they do know that it requires a higher level of prayer life in order to sustain a life of celibacy. That higher level of prayer life gives the priests the strength to serve our Lord, it doesn't make him any better human being than any other.