That was, I believe, the argument of St. Thomas Aquinas, which is not without a flaw because it relies on unwarranted assumpton that deity is not subject to regression.
There are certainly objections to Aquinas's Cosmological Argument (some I'm even fond of), but this wouldn't be one - by definition.
The problem of infinite regress is, briefly:
Each event must have a cause which must have a cause which must have a cause If this goes on to infinity then you are always at an event which must have a cause, and never get its cause. You infinitely stay at an event for which there is no cause. And therefore all the other events in the chain of causation don't get caused. Every event is in a chain of events which has no cause.
Therefore there are no events. The logic against this is: since there are events, any argument resulting in infinite regress is false.
The First Cause (which Aquinas says is God) eliminates this infinite regress because it is - by definition - uncaused. It requires no previous cause, no chain, and therefore does not infinitely regress.