So, if you weren't restrained by your belief in God you'd feel competely justified in doing whatever immoral behaviour entered your head. To quote you, "It doesn't matter at all" if God doesn't exist. Can you please notify your local police department of your lack of an innate moral sense, in case you ever get a crisis of faith. I sure hope that I don't live near you, I'd be really worried that something might shake your faith in God and then you might murder me if you wanted my possessions or my wife, as apparently your faith is the only thing stopping you from such acts.
The numbers of folks doing these things, compared with the size of the general population, is relatively small, and the effects can be "absorbed" on that scale (though on the personal scale it still leads to lack of trust and ostracization from the social group). No society would survive if such were taking place on a large scale, which is why societies have codes (most not Divinely inspired) to discourage such actions.