Maybe being a pacifist after urging the creation of the a-bomb is contradicting himself, but there's no contradiction inherent in being an ethical person and not believing in a personal God or the afterlife.
I really think that Einstein believed in God and had a huge dilemma trying to balance it with his intellect, because for guy that did not believe in God, he sure reference Him a lot.
Here are some collected Einstein quotes on religion.
A man's ethical behavior should be based effectually on sympathy, education, and social ties and needs; no religious basis is necessary. Man would indeed be in a poor way if he had to be restrained by fear of punishment and hope of reward after death.
It was, of course, a lie what you read about my religious convictions, a lie which is being systematically repeated. I do not believe in a personal God and I have never denied this but have expressed it clearly. If something is in me which can be called religious then it is the unbounded admiration for the structure of the world so far as our science can reveal it.
I think he was perhaps a bit conflicted about God and religion, searching for some mystical, cosmic organizing presence in the natural world, but he stuck to his guns.