I think you got your answer. Thats his way of saying no, without directly saying no.
He’s a very smat doctor. Knew I’d never say “may doctor told me not to get a vax” which, I think, could cause him some problems.