The oxygen masks are basically for passenger comfort. Passengers can be exposed for two full minutes to complete depressurization. This is from the FAA. They also specify (like they would need to anyway) that this requires an emergency descent.
It’s critical that the flight crew mask up fast. But the passengers can sleep and will wake up fine after the emergency descent.
Reminds me of a Southwest Airlines flight I once took. During the safety lecture before the flight took off, we were told how to put on the oxygen masks, and if you're flying with a small child, to put your mask on first, and then help your child. And if you're flying with more than one child, decide in advance which is your favorite!
Mark