Correct. Just like I'm not pro-racist but I am pro-free speech.
It would not be immoral to suppress racist speech, but it might be imprudent.
Racist speech is immoral, just as murder is immoral. However, for prudential reasons, we tolerate this evil because we, as a society, judge the cost to society of supressing this evil to be greater than the evil itself.
One cost of this approach, however, is the promulgation of indifferentism throughout society regarding evil speech. "Hey, I can say whatever I want!"