I think the etiquette guides all say that one should never decline a White House invitation to a state dinner. I'm not so sure that it applies to any invitation to do anything from the White House (an invitation to Joe's Crab in Miami). Of course, it's been many years since I remember reading something about this and have become much more uncouth (hence the name) since then.
You are correct, and I misread. I thought Saban had turned down a White House dinner, not a Joe's Crab Shack dinner.
So my comments about impropriety are interesting but irrelevant.