Then I suppose you've never had that particular remark directed at someone you love. I don't care if it is a pun, it's a very insensitive word. If he'd made some sort of a pun featuring, oh, "Jap", would that be funny, too?
A word is only offensive if you let it offend you.