I can understand your feelings. But throughout history every generation has thought that it was the best there was, and that the world was going to go to heck after them. Somehow the world kept moving.
Kids are hard, and it takes patience and sacrifice to raise them. Many today do not have those qualities. Sounds like you both realize that you might not, and have chosen to not have kids.
I have no kids, but I do not think of it as an 'inadequacy.' Sirloin, I think you are too hard on yourself.
1. I'm a full time college teacher, 80 hours a week, and it's a big job in my view, to help really screwed-up immature high school & college kids develop some character and personal responsibility. I clean up what the parents, many of them with a few divorces under their belts, have done (or not done) for their children. If (my husband &) I had kids, it would be extremely time consuming to do well - and I could not help make a big dent in the problems of delinquency and irresponsibility I help to correct every year with hundredss of young people in their teens and twenties. So, I'm very happy with my decision. Much happier, than if I'd had 1 or 2 children. You can't do everything well (have kids, and work full time). You must make choices.
2. We have 6.5 Billion people on the planet and with that has come stressed economic resources, crowded schools, housing shortages. I think we have an implicit "no vacancy" sign out at the moment -- population growth is a serious matter which we should all think about seriously.
Those WITH kids, especially with more than 1 or 2, could be viewed as the "selfish ones" in this light. Just a few more kids REALLY puts demand pressure on housing, living costs, and etc. in any community. "Thanks," for that...