If that was wrong to say, the implication is clear: even though a demonstration of this sort is legally permissible, it is morally wrong. Association with anyone of this group is de facto a wrong of the highest order, even under pretense of protest against taking down a statue. Is it not the crime of the century? The fact that former friends hold this view shows what dire straits we are in.
Guilty by a semblance of association.
I am against taking down any Confederate statue. So do you think I am a racist? So do you?