Coyne is,..., well, an idiot. Theists do not admit that God had a beginning. Physicists do not accept the steady state universe. IOW, materialistic science accepts as axiomatic, that all things scientific have a material cause. Atheists, then would seem to accept it as axiomatic that all things have a material cause, lest there be wiggle room for supernatural things, namely a god of any sort. So the best thing for atheists is an eternal bubbling super-universe where "normal" universes pop in an out of existence. IOW a poor man's god.
Nonetheless, it is a legitimate question to ask for the cause of matter. To which they would reply, "it needs no cause, it just is". Similarly when asked what God did before he created everything, the answer is, "He did godly things."
Very last sentence...LOL!