More fundamentally, marriage is what makes it possible for children to be raised by their biological mother and father. The reason that adultery is defined in Exodus specifically in terms of a man having relations with another man's wife (as opposed to a married man having relations with another woman) is that adultery, as defined, interferes with another man's right to know with certainty the identity of his biological offspring. By contrast, no matter how much a woman's husband sleeps around, she'll have no trouble knowing the identity of her children's mother. To be sure, a man who sleeps around may as a consequence of his actions not be certain who all his children might be, but since the victim and perpetrator of such harm are one and the same individual, there's no need for additional punishment.
While it's true that contraception and adoption change the equation slightly, for society to thrive, male-female mating couples need to be the norm. Children are most likely to grow up to participate in such couples if they have a parent of their own sex to serve as a role model for how they should behave in a relationship, and a parent of the opposite sex as a role model for what they should expect in a mate. A same-sex couple who adopts a child would deprive it of one or the other role model. In the event that there were a shortage of prospective adoptive families, such deprivation might not be the worst thing in the world, but it would be a legitimate basis for awarding male-female married couples priority placement.