In Rudy's defense, he was the host of a military-themed Survivor clone called "Combat Missions" back in 2003. The show itself was pretty good and while Rudy wasn't a great host, he was adequate.
Rupert might have potential as a sub-Survivor host, but I agree that Boston Rob would be a terrible choice.