Yep. They never came off unless he was in bed listening to unearthly sobbing.
I did Google that scene with the suit and the sailboat, and it was as I said -— plus Gail Russell had on a hat, coat and heels ON THE BOAT!
At the time seeing it — in ‘44 — it never seemed odd. I guess we expected everyone to be dressed ‘’properly’’ all the time.