It doesn’t have to be mentioned. If there was no sacramental marriage, the second apparent marriage is the first sacramental marriage. That is what annulment is, a finding that there was no sacramental marriage.
Now you are confusing the actual adultery with the intention to commit adultery right from the very first. The intention, to all sensible people, would annul the marriage. The adultery itself would be grounds for putting the person away but not for remarrying, if you accept the fact that the Gospel doesn't discuss, and therefore doesn't allow, lawful remarrying in this case.
How can anyone know about the intention from the first? So how can other people make the decision?