Not completely true.
She could resign like Newt did. Some scandal could cause her to resign the Speakership.
There could be for some reason a change in the majority of the House at which time the new majority would force a vote.
If the Dims loose confidence in her leadership enough that a large number of the Dims in the house circulate a petition calling for a vote for the Speakership she could be voted out mid term.
We could not get Clintoon and Pelosi has all sorts of scandals that never touched her.
If health care fails, it's technically possible, but they would more likely blame Bam-Bam.