Well, this screen grab from "Goldfinger" says that British super spies have been known to experiment with rompers...
That’s more “Hey, I’m either getting out of or going to the Pool” wear, and much much LESS “Hey look at me, I’m too avant garde for mere words” wear.
And that’s Bond. He can wear whatever he wants...