The Nazis had a major structural impediment to "getting The Bomb."
It is based on Jewish Science, you know. A significant majority of top physicists had fled Germany because they were Jewish or leftist (true, unfortunately.) A counter-productive policy for the Nazis if there ever was one.
It was not as if Jews everywhere were coming up with revolutionary scientific breakthroughs- it could be specifically narrowed down to German Jews who had been educated under the German system, Jewish or not; who became the scientific revolutionaries of the period. You could stretch that to include the German-centric cultures of Switzerland, Austria, etc.
Hitler committed a blunder by turning his madness onto the Jews; but even without them, had Germany not gotten into war, the Bomb would have come into their possession- it was just a matter of time. Others would not have progressed as far as they did, had the German scientists not been forced to emigrate.
Besides that, the pressure of war accelerated developments outside Germany. Without that pressure, and assuming peace to have prevailed in Germany, they most likely would have gotten the Bomb first, and with it, probably the only country with a capable missile system to deliver it reliably.
The war was a boon to the rest of the world, in a sadistic way.
Edward Teller (an Hungarian Gentile) attributes it to other cultural factors in the German scientific community. Fission was certainly discovered by a German and there were plenty of German physicists who knew what it would take to make a bomb. (The Germans spent more on the militarily useless V2 than the Americans spent on the Manhattan project, so they did not fail for lack of resources.)
As Teller tells it, early attempts at making a nuclear reactor were fustrated because normal commercial grade carbon is contaminated with boron, a neutron absorber. Leo Szilard, another Hungarian physicist, who had been a chemist, and was at Columbia at the time, recognized this and the Americans (and Fermi, also a Gentile) succeeded in making a nuclear reactor by using far more expensive reagent grade carbon. Teller claims that Heisenberg, who was leading the German efforts, would never have talked to a chemist.