In both cases Panspermia's advocates say there's simply not enough evidence to demonstrate that life could originate on Earth given that it is so small and inconsequential, or that the mutations necessary for evolution can occur in such a limited environment as is available on Earth.
There are, of course, variations on those points of view.
Panspermia happens to be consistent with the traditional view which has God bringing life to Earth from other sources. It is not, however, consistent with the geocentrism inherent in current evolutionary theories which require that everything take place on Earth. Further, it is not consistent with the traditionalist viewpoint that Earth was created only a few thousand years ago.
Presumably, someday, we'll understand enough about life to figure out how it works.
Thanks for the explanations and the introduction to Intelligent Design!!!