.
A more obvious and pertinent question is : When is a ‘Speaker of the House’ unfit to serve, and should be removed.
.
A 2/3 vote of the House would do it per Article 1 Section 5 of the Constitution. That’s about as likely to happen as the removal of Trump from office via the 25th Amendment though.