It is often quite pronounced.
I got that same feeling the first time I watched O'Reilly on some talk show or whatever it was. I just took a dislike to him. Couldn't explain it.
I also got the same negative feeling about Bill Cosby. I loved hime when I grew up and listened to his records. But in the times I saw him on television, something hit me very negatively. Could never put my finger on it. But when I heard many years later that he was accused of drugging females to have sex with him, it didn't even make me bat an eyelash. I don't know if it is true or false, but my gut tells me he could do that.
As for O'Reilly, when I saw his infamous "F*** it! We'll do it live!" video, I realized I had hit the nail on the head with my gut impression.
As a tv personality, Cosby used to put too fine a point on... everything, with a little too much self-conscious flair.