Actually from my study of history that was never the case. At the beginning of the twentieth century there was a bordello on every street corner in New York and in many other cities across the US.
It really comes down to basic biology. The sex drive is powered by testosterone and men have 8-10 times more testosterone than women. So on average men want to have sex much more often than women and if their wife won’t satisfy that need then many men will fulfill it in some way.
Not saying it’s right, because morally it’s not, but it is what it is.
I said “patriarchal gender norm,” not “universal practice.”
People decided that, since the norm wasn’t universally observed, we could just do without norms entirely. It hasn’t worked out very well, as anyone with two functioning synapses could have predicted.