A wise observation. Now that I'm retired, going at 9am means little crowding and a fair number of cute soccer moms. (my WM is in an upscale burb).
Before I retired I'd go at 9pm. Little crowding but no moms.
At peak times, yes it's a pain in the @ss.
Whether I go hiking, to the beach, sometimes even to work - it is always worth going earlier. The people I don’t want to see are sound asleep in their beds while I’m enjoying myself/getting things done.
I have a very short commute; I don’t know how people tolerate other drivers for an hour at a stretch...