If he was just going to desert, he would have taken his weapon with him.
Bergdahl knew if he walked into that area carrying a weapon he would probably be shot on sight, so he left his weapon behind.