Because the same people say the universe must have had a beginning in straight-line time and couldn't have always existed.
It is not special pleading because it is a category error to ask where an uncaused first cause came from. A necessary being, by definition is not a contingent being. A necessary being is eternal and uncaused,
Now you may try to assign these attributes to the universe itself, but that leads to problems of explaining things like the illusion of cause & effect and why the universe has an apparent beginning, or if the series of past events never began to exist and there were an actually infinite number of past events, how we arrived at the present moment.
Cordially,