Agreed.
But some leading libertarians are not so willing to leave it to the states. Pat Buchanan wrote an article not too long ago calling for a truce in the Culture War, in which he basically called for a states rights approach to social issues, including marriage. But "Reason" magazine's Nick Gillespie took issue with that by taking the thorougly leftist, thorougly absurd, position that such a truce is not acceptable because the Constitution demands abortion and gay marriage rights, and as such, the unenlightened masses of backward states must bow to a wiser rule of federal judges.
I disagree with Gillespe. Abortion and gay marriage are no more a federal issue than legalized gambling, and both issues should be left to the states. The only exceptions that I could see to that would be issues of racisim or religious descrimination which are urgent enough to require federal intervention/mediation.
Obviously, Gillespie does not speak for all libertarians, but he is one of the most visible mouthpieces for libertarianism on television. Still, I may be giving him too much credit for influence among libertarians, but it would be nice to hear more libertarians take your stated position.
Speaking for Libertarians is sort of like speaking for a herd of cats: You might be making the most noise, but we're pretty much going to do our own thing regardless.
The modern Libertarian position really formed itself around the later career of Barry Goldwater, who's committment to small government and individual rights was an excellent example of the Libertarian ideal.
I don't think I disagree with anything you say. As a nation, we have elevated protections based on race and religion to the Constitutional level, so one can make a case for federal intervention (leaving aside the whole debate about the Incorporation Clause of the 14th Amendment). At no point, however, have the people consciously given consent to the idea that issues at the center of the Culture War (abortion, marriage, etc) rate as inalienable rights beyond democratic/popular control. And its certainly not the role of judges to arbitrarily say otherwise.