I know quite a few conservative atheists who feel the same way.
The fundamental link between conservatives is the shared Judeo-Christian morality that underpins their politics...I am not sure you truly can be a conservative atheist. That is not to say that you cannot be on the right side of politics, but just that you cannot be truly a conservative.