Yep, they don’t want any part of him as Speaker.
That they didn’t launch this attack when he was doing hard questioning but only after rumors of him becoming Speaker came out tells us that’s what they really fear. They may have succeeded in killing his chances, depending on whether enough members of Congress cave.