Ya know! it is really easy for someone on the outside looking in to say he should have said this or said that, when in reality no one replies perfectly.
You're right, few people can reply smoothly at all times. But losing your temper prevents you from thinking clearly.
As I said, my model is Mr. Fairview, who has been verbally sucker-punched this way in public situations on a number of occasions. He smiles and answers courteously, and then the questioner looks down and sees that his guts are on the floor. It's a beautiful thing to see.