That's not as clear cut as Will says it is. Because, if he has not broken the marriage contract, fidelity and all with his co-habitator and kids, then the contract must have been null and void based on her state of incapacitation. If on the other hand his infidelity did break the contract he can't be considered sole arbiter. They can't have it both ways, IMO.
In law, a husband and wife are "one" until they are no longer husband and wife, in law. All the external factors you list have no bearing on the legal view of husband and wife.
Where is the law that supports your contention?