Sure they do. Furthermore, such inferences camouflage the fact conventional "science" threw in the towel on abiogenesis a long time ago.
What's more if you can't confirm your guesstimates, what makes you so sure Fermi estimates are any better than the guy looking for his wallet under the street lamp, because the area he lost it in was dark?
Finally, unconfirmed, as opposed to disputed, planets are nowhere near analogous to attempts to make "life" just another mundane physical process.
Showing me a bird on a fence post does zero to explain the turtle, even if it's the exact same fence post.
The sad fact is this area of science is tainted in the same way as "gun control" advocacy; forget whether the thesis is verified, plausible is good enough....just assume it's correct and legislate accordingly.