I have thought that too, so I know where you're coming from. The problem is, that we tend to look as Heaven as a right that has to be rejected by sin, rather than what it really is, a privilege and gift that cannot be earned.
Go back to the Latin definition. Heaven is "supernatural" happiness = super + natura, "above our nature". We can no more naturally live in heaven than a fish can live in the air. Remember all those Biblical passages like "no one can see the face of God and live"? Well, heaven is seeing the face of God. That vision is so majestic it would literally destroy us, unless God granted us the special capacity to withstand it.
For a man to go to heaven, he must be made a new creation. He must be given divine life and the capacity above his own natural existence to see the face of God. This is what the sacraments give us ordinarily, and God can give us extraordinarly even outside of the sacraments.
If a man does not have this supernatural gift, he can still be happy. He can be happy in having enough food, company of his friends, pleasure in his body and the absence of pain. This is Limbo. So when we place babies or unbaptized folks in Limbo, we are not saying they necessarily have to suffer. They may well not suffer at all. They just, in the scheme of things, experience a lesser happiness than the souls in heaven.
And Catholic theology is clear that there are degrees of beatitude. I'm not going to have the supernatural happiness of, say, the Blessed Mother, or St. Francis.
Yep, which is why I have never believed in 'Limbo'.