Occam’s Razor Answer: They feared that the not-small-number of anti-Semites living in their own countries would have looked at that and said “Oh, yeah, cool!”, turning them pro-Nazi and thus undermining their war effort.
I don’t think that at all. I think they didn’t want their plan and mission complicated by emotion. Cruel? Maybe....
I don’t believe that.
It’s one thing to not allow a group in a country club, it’s very different thing to round up that group and systematically gas them.