Not if only 80% of the voters actually use the highway, and if the highway can be funded by using the taxes from only 80% of the voters. A libertarian who votes against the highway and doesn't pay the tax on it and then uses it after it's constructed is acting immorally because he's using something he didn't pay for. That would seem to be fraud or theft in my book.
Wouldn't we spend more enforcing who can use the highway?
I cannot imagine a system of government working that way. How would you even begin to keep track of who voted for what? I think State and Local peoples have more flexibility than that, even if it means someone is "force" to pay a tax. There are many ways to ensure balance. We just need to get Government back to the people.