See National Socialist Party of America v. Village of Skokie,
https://en.wikipedia.org/wiki/National_Socialist_Party_of_America_v._Village_of_Skokie
Yep - I don’t think it’s illegal to wear a Nazi shirt or march in Skokie (just obnoxious and something you’ll probably get yelled at for), but some Smith Act provisions seem to survive First Amendment challenges and the Smith Act has certainly been construed as applying to Nazis in the past.