The difficulty comes with the fact that if same-sex couples can claim that they are "married", they will almost certainly be able to use this claim to force private companies and private/state/local institutions to treat them as any other married couple. This is the overt intention of some of the groups pushing for "gay marriage"; the threat is hardly imaginary.
One of the things that makes any society strong is the notion of "kin"--a set of people with whom one has a blood relationship which is irrevokable. There is a set of people whom you will help out in time of need because you are related to them. Although you may decide to stop supporting a brother because you don't think enabling his bad behavior will help him until he solves his problems on his own, he would still always continue to be your brother.
If a married man and woman adopt a child, they become the baby's mother and father. Although not biologically related, the child's place in the family is no different from a biological child of those parents. The mother's sister is still che child's aunt, and the aunt's child is still the child's cousin. As far as anyone is concerned, the married couple are the child's legitimate mother and father.
One of the key factors that distinguishes a legitimate child from a bastard is having an identifiable mother (biological or otherwise) and an identifiable father. Even if a mother or father is no longer living or no longer with the child, the identity of that person forms a key part of the child's identity. A child adopted by two women will be fatherless, and a child adopted by two men will be motherless. How can such adoption--except in cases where the child has acknowledged and continues to acknowledge his parents (e.g. a twelve-year-old who gets orphaned and is adopted by his uncle--who continues to be his uncle and not his father) not create bastards?
I don't think homosexual marriage is an issue of less government- it's an issue of more government. This push is being made by forces that inevitably want more taxpayer money and more regulaions on citizens.
I can't think of an issue that is less time-wasting than defense of the first ammendment. If you truly believe in less government, why would you be in favor of the government micromanaging what we think, what we say and what we believe.