The 9th Circuit Court of Appeals judges were not voted into their current positions. The President appoints them, and, with the consent of the Congress, they "win" their place in the court.
If punchcard ballots are good enough to elect Davis, they're good enough to recall him.