Married with one little girl (haven't you seen the pictures of my little Hannah posted a few times on these threads? I could post more! ;o) and another due in February.
Good analogy. But (that word again ... sheesh) disappointing God is a lot different than turning our backs on God. A sin is a willful disobedience to God, is it not? Likewise, if a spouse is showing willful disobedience (such as an affair or something), is a denial of love.
We may disappoint God by (for example) not praying much, not going out of our way for the needy, etc., but we downright deny Him by committing sin against Him. We then run the risk of being told "I do not know you" when our judgment day comes.