“As for a sizable enough asteroid destroying all life on earth, the chances are unimaginably remote. Preparations for such an event would also be feasible (shelters, etc.).”
You missed the point. It isn’t the asteroid which destroys all or most life on Earth that you have to worry about. It is the smaller asteroid which is capable of destroying the present level of global civilization. Once this civilization is destroyed, there will be insufficient time and natural resources to create another such spacefaring civilization before the Extinction Level Events (ELE) occur. Finally, even if the Human cultures stranded upon the Earth managed to survive that long, the Earth’s still thinning atmosphere and the Sun’s expansion to become a red giant star will eventually reduce the Earth to a lifeless or virtually lifeless planet stripped of its hydrosphere, upper lithosphere, and atmosphere, if the Earth survives at all. The sun will eventually expand to vaporize and consume Mercury and Venus. The Earth may or may not escape the same fate, but would be in the outer edges of the Sun’s atmosphere even though its orbit has expanded outwards to the vicinity of where Mars is today.
These smaller asteroids impact the Earth less than about 500,000 years on average. Smaller asteroids capable of major disruptions of the global civilization occur about 100,000 or more years on average. The Earth can expect one of these civilization shattering impact events to occur anytime within the next few thousands of years or tomorrow.
Note, if Comet Shoemaker-Levy 9 had impacted the Earth instead of Jupiter, our present civilization would already have been burned off of the face of the Earth along with all surface plants.