I think your point is a very good one. It's not Hastert could remove Foley from office. What exactly was he supposed to do if there are just rumors? All he could do would be to talk to Foley and tell him to knock it off.
Now if there was any evidence of Foley making sexual advances toward underage pages then he could have contacted law enforcement and then push for censure.
There is no way Hastert could have done any of that just by the content of the emails and some talk about "being creeped out".
you know what he could have done? Brought in a Dem to ensure that it wasn't a one-party problem.