It would appear that there is some untapped political opinion out there, no? You bet there is. That's why the Libertarians flock to Free Republic. There is a lot of sympathy here and elsewhere for the general philosophy, it's just the specific proposals, or some of them anyway, that sound kinda loony, or, at the very least, unrealistic, know what I mean?
You asked what will keep it from sinking to insignificance. When nearly half the population of a very liberal(/socialist/communist) state supports a proposal that both major parties oppose, the sinking to insignificance would appear to be prevented, no?