No we can't. Federal Judges can be impeached for cause by Congress. They can also be told to dissociate themselves from certain cases by Congress. Neither will happen because they have neither the principle nor will to take on the Feral Judiciary at this time in history.
Our only recourse is to replace activists with Constitutionalists but here again our Congress folk don't even have the wherewithal to get a fine man like Miguel Estrada a vote on the floor of the senate.
Seems like maybe our Congress is the root of the problem.