No, this guy has problems. If he had teased back and said “which one of you is Hermione and which one’s Ginny” or something like that, it would have shut the two girls up.
True, but some people are too stupid to think of such witty retorts, I’m guessing this chap is one of them...