A flaw in your argument, Charles didn't go back to Camilla after several years and two children, he was with her the night before they got married.
There is no source anywhere saying they were sleeping together that nite, which is what you seem to infer. She was married at that time, and their affair had ended pre-Diana (while Camilla was not married). That doesn't mean they didn't still care for each other; obviously they did, she gave him the infamous cufflinks. As I stated in my reply earlier, if circumstances would have been different they should have been together from the start. But it didn't work out that way. Life is funny.