YES!!!
However, no atheist will admit this. They call claim to believe that somewhere outside the universe is floating a gigantic granite block with "thou shalt not be a bigot" carved on it.
G.K. Chesterton once said that the problem with those who reject religion isn't that they believe in nothing, but that they are willing to believe anything.
Most people I know who rejected the faiths they were raised in didn't become rational scientists, they became the "I'm spiritual, not religious" types who believe in all sorts of New Age mumbo-jumbo (pseudo-Buddhism, pseudo-American Indian shamanism, magic healing crystals, etc). And, as you say, they replace conventional morality not with nihilism or utilitarianism, but with doctrinaire political correctness, with its own invented sins of "racism, "sexism", "homophobia", etc.
People have an instinctive need for ritual, belief, and moral codes. Take one away, they'll find something even more nonsensical to fill in the gaps.