I imagine he has.
The Speaker of the House assigns and re-assigns.
No mystery really.
I imagine he hasn’t. So I’m going to say he has NEVER removed someone from a committee for having a too liberal a voting record and challenge someone to prove me wrong.