Felons are not a race.
It isnt racist if you reject felons of all races.
You might be able to claim they are criminalists. Which is just God-given common sense in action.
Let the folks who say its racist rent the felons a room in their own house.
Maybe criminophobes is better than criminalists.