In fact, I see it as a clever answer. He graciously avoided attacking Barbara Bush directly and instead turned it into an attack on her two retarded sons.
“... He graciously avoided attacking Barbara Bush ....”
That’s how I read it too.
IOW, “Of course the Mom is going to be pissed at me, it’s understandable, since I stepped all over her kids.”