Taste has nothing to do with it. He is the head coach of a professional team that is in go mode right now. As a man who used to be part owner of a baseball franchise Bush is likely to have understood Saban's decision.
I'll retract my propriety statement, since it wasn't in fact dinner AT the White House that Saban turned down.
I still think he missed the chance of a lifetime, though, and if I were his wife I'd probably give him a "What were you thinking?" arm smack.