The word in greek isn't adultry it's pornea and it means sexual immorality. There's a wealth of things which could constitute that. In fact Christ didn't even use the common greek word for adultry, which is a different word, he quite specifically used pornea.
But where does 'pornea' even begin to include where someone just gets tired of their spouse and wants a divorce? That is the situation that I often see for anullments here.