Only by resigning. He's toast, and should be considered as such.
I despise him so much, that if by some miracle he got the Pubbie nomination, I'd tell my friends in SC to vote for the Rat, just to get rid of Goober.
I think you will be stuck with him another term at least. Hopefully he does not make you upset too much. lol.