They can kick out the Speaker anytime there are enough votes to call for a new Speaker. AND remember the Speaker does
NOT have to be a member of the US Congress. Newt's name was bandied about when the "son of a bartender" quit abrupt(Zero's golfing buddie), and Paul Rino filled the gap. It was said the Freedom Caucus could not decide who they wanted and Paul Rino first said he didn't want it and wouldn't accept the nomination unless his three demands were met (one was no weekend work - hell they don't put in 40 now!) - then Rino backed down and capitulated.