The reference is a little vague but Harry could have meant the private residence
https://archive.org/details/presidentialcour00besc/page/210/mode/2up