He probably accepted the invitation out of respect for the office. If he had been a Democrat who had been invited by Bush, he would have made a huge show out of publicly declining and trying to embarrass the President.
In other words, Sully, unlike the Democrats, has class.
I suspect if he had declined, no one would even have known he was invited....and certainly the discreet Sully would never have mentioned it.