Yes, the pointing was impolite, but his habit of making all those crazy hand gestures was completely off-putting.
Maybe he's nervous because he believes he's tanking...:)
I hear people gesture excessively when they want to appear "believable" although they know they're lying.