Now he's upset? Bet he thought he was being hip at the time.
If you follow the link, you can see the whole segment. He does look uncomfortable. He should have prepared himself for what Colbert is about.