We must be from the same generation, because I also grew up with the Nelson's and AB. I don't know who raised the children, but they seemed to totally avoid the topic of their mother on the King show. This article says Kris gained custody so maybe they did grow up with her. I remember she was a very attractive woman. They did mention on KIng that she had remarried, but didn't elaborate on it. Maybe it's a sore subject with them.
Tracy and her brothers pretty much raised themselves with Rick on the road a lot and their mother "not always available to them." Turned out pretty good, actually... I think the kids were all really close to their grandparents and Aunts and Uncles growing up.
It was pure coincidence that I just happen to have bought Rick Nelson's Greatest Hits CD this past week. When I was a kid, the most important thing in the world was to see the end of Ozzie and Harriet so as to not miss "Ricky" and his band. One interesting story from the CD's liner note is that his hit song "Travellin' Man" was originally pitched to Sam Cooke. It's really easy to see why when you think about it.