In the uniform code of military justice (UCMJ), adultery is a chargeable offense which can be punished by jail. The UCMJ is established by the US Congress.
Oh, adultery is a crime in some states. I'm not denying that. I'm saying that if it were a breach of contract, it wouldn't be a crime.