There isn't a libertarian argument against homosexuals marrying, the libertarian position is gay equality, including in marriage, the military, adoption,etc.
The state and religions, and tribes have always defined marriage, you just didn't notice it, but if you showed up in 1790 and wanted your marriage recognized by the Army, or for a marriage license, you and your boyfriend wouldn't make the cut, nor in ancient Rome, or Greece, or when the Catholic church made marriage law.
The fact that a "marriage" between "Adam and Steve" would not have been recognized by the any of the several states in 1790 or the Latin Church (or for that matter the Orthodox Church) or the Roman Empire, does not mean that those institutions defined marriage, but that they recognized what marriage is, rather than arrogating to themselves the purported authority to redefine it. A married couple arriving in the Roman Empire from the Persian Empire or the lands of the Scythians would be understood to be married, even though neither the Roman civil authorities or, in latter times, the Orthodox Church, had married them. A couple married by the Church (East or West) travelling along the silk road to China would have been understood to be married by all the various cultures, religions and societies they met.
The notion that regulation or registration by the state makes something into a state institution, defined by the state, able to be modified by the state, is a profoundly statist notion, and as such anti-libertarian.