A big part of the problem is that conservatives roll over and take it when they are attacked for being racist. Look at the last couple Speakers of the House.
The libs play Whack-A-Mole with conservatives. A conservative makes a perfectly appropriate statement that can be twisted and perverted and is then presented to the public 24 hours a day as proof that the conservative is a racist. The conservative resigns. The libs cheer.
You are exactly right.
Yes, Conservatives set themselves up as easy targets when they spout purist ideologue sttatements as if they were perfect people. They box themselves in with no easy way out.