"Physical harm" in the light of day, probably not.
But hate-groups make members bolder about their bigotry and make their target victims either scared, angry or both (fight or flight). I think that it does increase the likelihood of a confrontation of some kind by somebody.
If you are a Jew and see people openly celebrating Hitler, or you are Black and see the Klu Klux Klan partying in the open and celebrating their evil deeds, it does have an affect on you whether you want it to or not.
In America at least, we put up with it. And who in their right mind would make themselves a social outcast by joining the KKK? They defeat themselves.
I'm much more bothered by people who think that these groups should be banned or somehow censored.
Those are good observations, and I agree that drawing attention to this incident lets people know that such trash are out there.