The unstated premise in this byplay is that the atheist knows good and well that God exists, but refuses to acknowledge that fact.
Rom 1:18 ¶ For the wrath of God is revealed from heaven against all ungodliness and unrighteousness of men, who hold the truth in unrighteousness;Rom 1:19 Because that which may be known of God is manifest in them; for God hath shewed [it] unto them.
Rom 1:20 For the invisible things of him from the creation of the world are clearly seen, being understood by the things that are made, [even] his eternal power and Godhead; so that they are without excuse:
Rom 1:21 Because that, when they knew God, they glorified [him] not as God, neither were thankful; but became vain in their imaginations, and their foolish heart was darkened.
In the vernacular, what this says is that every man knows God exists, because God has shown it to them in the world around them (the inference is so simple they are without excuse).
BUT, and this is a very big "but,' if they should refuse to acknowledge the obvious, he gives them up to believing (or more properly, being dedicated to) their own excuse.
And yes, that does mean "evolution" for the "modern" excuse.
Well, that's not the usual meaning of the word 'atheist'. Defiance and disbelief are different things. I think Darwin was sincere in what he said about his loss of belief and becoming -- if not an atheist -- an agnostic. Here's the previously cited link to a Christian source that describes it.