Because legislation properly concerns violations of individual rights.
When we start legislating morality, the question becaomes "Whose?"
Shall we all be required to keep kosher?
Shall we stone adulterers?
Shall we imprison people who harm cows?
Shall women be whipped for not wearing chador?
Libertarians also argue you should be allowed to freely associate, or not associate as you wish. All laws regarding discrimination, except those prohibiting government discrimination, should be repealed.
Often taken to the anarchic with no concern for societal rights.