I would have said, “It’s a mandatory question? Says who?”
I would have liked to heard his answer.
WRONG.
You’d be getting sucked in to the game, because now you are debating whether the question is mandatory or not. Who cares if the question is mandatory?
The answer is “It may be mandatory for you to ask it, but I refuse to answer it.”.
Too many people here think the solution is to get cute and clever with word games. It isn’t. The solution is to refuse. Hit them head on. It can be done politely but firmly. Just Say No.