Not necessarily. However, I happen to think that Pat Buchanan -- whose opposition to "neo-cons" has a long history -- is in fact an anti-semite.
And I think those who tend to use "neo-con," as a perjorative tend also to be anti-Israel, though that does not necessarily equate to anti-semitism. But there does seem often to be a "Jewish Cabal" element to their complaints.
For an interesting take, much better stated than mine, see this article about the supposed Jewish Neo-con Trotskyites (about whom Ron Paul had so much to say, the other day).
Then you ought to refrain from saying that some use the term "neocon," because "it's a polite way for them to say 'jew-boy communist.'"
You need a stronger case than that to demonstrate anti-Semitism. Have more care, because false charges of racism are generally least flattering for the accusers.