It came across to me as if he was justifying walking out on his family. I was going to be magnanimous and avoid bringing this up, but since you already did ...
I think that at the time he broke up with his wife, he decided not to contest her getting custody of their daughter. I get the impression that he saw no point in having much contact with his first wife.
I was also troubled by him seemingly walking out on his daughter... but I was glad to hear that they finally did work out their relationship.