Foley was ousted as speaker because he lost his re-election to Congress. It had nothing to do with the votes in his caucus.
Newt was the next Speaker.