My relatives all remove and replace plugs, fixtures and such without turning the power off. I think they’re nuts but they don’t ask me.
That’s perfectly safe if you understand what you’re doing.