To be more clear, an appeals court can remove him from the case. He could even be reassigned to get fewer cases maybe. But he can’t be fired he can only be impeached and removed.
He can die too.
Sad that a judge can’t be removed for misconduct and abuse of power. So I would reassign him to a plot in the middle of Death Valley.