They dont even need a key. I have watched flight attendants lock and unlock doors from the outside with a hidden latch.
You are correct. Lavatory doors do not have a key lock. From the inside you slide the deadbolt latch which secures the door closed and brightens the Lav lights. From the outside there is not a keyhole. There is a hidden latch as you stated. Crew rest compartment doors have a key for access. Also, there is no ignition key to start the engines.