He reportedly got a lot of flack from the Black Caucus in the House for not joining in on the attacks.