You've got something there. Vitter was not part of the GOPe, as we define it, but he was definitely an old face in Louisiana politics. In a way, this race is a confirmation of the "year of the outsider" narrative.
Also, speaking objectively, Edwards' advertising campaign on "prostitutes over patriots" was devastating. The Democrat base may not care about such issues, but the Republican base does. It's a political reality, and railing against the base does no good at all. The R's may have been asked to hold their noses one too many times, especially when confronted with an attractive alternative.
The GOP base voted for Vitter. It was the Country Club contingent that abandoned him.
Also Vitter has been in national office for only sixteen years. We’ve had people around much, much longer than he, such as F. Edward Hebert, one of his House predecessors, and Senators Long, Johnston, and Ellender.