There are even more people in those homes who have children and grandchildren that don't visit than there are those who do. I would think that's a far more depressing feeling.
And having children with the assumption they will care for you when you are old is selfish. Some people do choose not to have kids because they are aware they are selfish people...but not all childfree people are. Everybody has their reasons as to why they do or don't want kids. Children are a never-ending responsibilty and even the best of parents have moments where they resent their kids.