Free time isn't the issue. It's being outta touch without anyone, presumably, knowing where you are when you're the governor of a state.
That said, I'll bet the WH knew where Sanford was. Someone should ask Gibbs.
That is why they have a Lt Gov. By the way, Sanford’s number two is the loudest critic of this episode. Eventhough they are both Pubbies, Bauer is a political rival and enemy. If the actual facts ever come out I bet his wife knew what he was up to but didn’t want to reveal it to a nosy press and self righteous public. One final thought; voters don’t own elected officials.