The key word there is "attempt".
While not all libertarians are unbelievers, I find quite a few are.
Quite a few conservatives are unbelievers.
I think it has to do with many libertarians' rejection of an authority above their own.
You seem to be saying the unbelief of some libertarians derives from their political views. But this is backward. They're atheists first, then reject authority over them, then become political libertarians. (Even if they do start with the political view and work back, the atheism is logically prior.) Christian libertarians start by accepting God's authority, and then wishing to see all lesser, temporary authorities submit to (and thus not try to take the place of) the great Sovereign of the universe.
My argument shall be extended from "It is immoral because God says it is " to "It is immoral because it infringes upon the well being of society" and that is where government should step in.
Two issues. First, why is it immoral to infringe on the well-being of society? Either you resort back to "God says so", or else it's baseless.
Second, who gets to define the well being of society, and who gets to decide when it's been infringed? The courts? The President? The majority? But the courts still uphold Roe, the President was Bill Clinton for eight years, and the majority of the people, I fear, are unregenerate. So who, and how do you know you can trust them not to decide that the Bible, being homophobic and sexist, infringes on the good of society?