take it one step further. We defeated Japan in WW2. Italy. Germany. Are we going to forbid them flying their flags too? This gets into the theater of absurdity if not something darker quite quickly.
You might get a debate on the nazi flag, but I don’t disagree with your premise.