If the libertarian party ever pulls more than 3-4% of the vote, then you might smell fear. What you're smelling now is indifference.
Come to New Hampshire with the Free State Project, and we'll see if we can make that happen.
I think you're right in the sense of being concerned about the Libertarian Party making a serious inroad on political offices. But I sense something deeper. I think there is fear, in the sense of being a fear of ideas. Statists really hate dissent of any sort. They wig out over the littlest things.
Remember the big brouhaha over the Dixie Chicks? That puzzled me because, after all, who really cares what any celebrity thinks? And it's not as if it changed anything; the Wargasm crowd still got their Splendid Little War®.
People with different ideas seem to be greatly feared merely for existing.