And we might not have managed it if our enemy hadn't conveniently driven out some of their best scientists.
Do you mean Italy or Germany? Had the NAZI's driven Heisenberg out of Germany, for instance, his erroneous atomic math probably would have been taken more seriously than the American's correct math (performed by native American Feinman, for instance).
Now Fermi from Italy, on the other hand, certainly gave us a two year or more head start.
But our guys would have done it either with or without the foreigners. Our math was correct, after all.