It effectively does when they are really awful and the press highlights them. Remember, conservatives argued that you can judge Obama by the company he keeps (Wright, Ayers, etc.) and that's what they are doing to us.
Obama has hardly banned Ayers and Wright.