Both. I think he believes (correctly) that if he answers the question then there will be those who say he can’t be objective in participating in the case and that ultimately he will have no choice but to disqualify himself.
But on what grounds? Because he he has his own idea of the definition of natural born citizen? That's not grounds for recusal.
He probably answered the question the way he did because he didn't want to be bothered by someone like Klayman.