No, they aren’t all evil. And a lot of felons aren’t evil, either, they broke the law.
But both felons and illegal immigrants are criminals.
I am not a lawyer. Can a lawyer (maybe you, if you are a lawyer) clarify exactly what the US Code means in this area?