The matrilinial descent rule for Judaism is as much nationalistic as religious. This, however, is not the source of the race business. Judaism and race is more of a form of epithet. It was/is used as a way to discriminate, ghetto-ize, and attack.
Well then, why can't they be Jews in the nationalistic sense?