The exchange is here:
http://www.c-spanvideo.org/program/293017-1
between the 1:13:28 hour mark and the 1:14:10 hour mark.
He also says to the committee chair, who was fishing for a nomination to the supreme court, “You don’t even have to be born here. We don’t ask you that.”
Thomas is a smart guy.