Throwing him under the bus could be even more damaging to the party. It reinforces stereotypes that average Republicans out there are engaging in some "war on women." It looks like the party found such a Republican and told him he's fired. Let him stick around, make things right, explain himself, and we can counter the stereotype.
He’s not so special that the political capital it would take to rehabilitate him (even if it were feasible to do so) would be worth it.
Are you really telling me that he’s so great that anyone who might replace him would be a vastly inferior pick?