I went through that line of reasoning in my late teens and early 20's. The fact is there is no moral right or wrong unless established and enforced by God or (for sake of arguement...there being no God....) the stronger more cohesive groups of humans who can forcefully impose their visions of morality on weaker groups of humans!
The book "Lord of the Flies" paints a dark and horrifying vision of what humans do when all restraints are taken from them! There again it's only dark and horrifying to my "point of view"....it may be idyllic to another's point of view.
The strongest "point" of view too often in our world is often an example of co-ercion via gun "point"!
Democratic Republics are fragile constructs...the World fears the US for not what it is but what it could become....Imagine Rome with Nukes and the ability to project lethal force as far as the moon when deemed expedient...that is what the World really fears!
So, by your line of reasoning, prisons should be full of atheists, right?