I doubt the President is too worried. Its his second term so he never has to worry about reelect again. As far as his legacy, all he has to do is look to Truman. Truman had to make alot of difficult, unpopular decisions and left office with approval ratings in the 20s and was very unpopular. Over the last 20 years of his life it became clear he was right and became a well-respected and liked former president. Now look at him, he's usually put in the top 10 in list of our best presidents.
He is foolish to not be worried. If this dynamic leads to a democrat house, they will spend 2007 impeaching him. He doesn't need that kind of protracted distraction, especially while fighting a war.