Sounds like you have not experienced anti-semitism. People who care about it can tell a Jew by his appearance and name, they think. And they pay attention to these things.
You are correct.
When a stranger taps you on the shoulder, and then, when you turn around, you catch a fist in the face -- hard -- accompanied by some short rant containing the phrase, "Jew-boy", uttered by a smirking chap, who walks off having done the deed, it does tend to color one's perspective on these matters.
At least, it did mine, when it happened to me.
I wish I could say that the experience was an outlier, a random "meaningless" event, however, it happened in a context rife with similar "experiences". This is what happens when one is growing up with "the wrong religion" in certain rough neighborhoods in the 1950s-1960s. Maybe even later, but someone else would need to weigh in on that matter. (All I can say is that the antisemitism I've experienced since leaving that context has been of a more "subtle" variety -- but no less real.)