Failure is stressful
Bingo. The economy is still in the tank, the jobs numbers are horrible, his internal polling shows bad news on the horizon, there's more trouble brewing in the middle east, he can't smoke in public etc. The Presidency is a tough position, but especially so when you're failing.