Theologically speaking, there is a way to look at children going to Heaven, as addressed by King David. Through the writings of David, it is believe he was saved (2 Samuel 22:2-3) and that when he died, he went to heaven (Psalms 23:6).
When David speaks of the death of his infant son, he speaks of going to him, but his son not returning to him (2 Samuel 12:23).
That’s a decent argument that carries some weight, and I tend to buy into it I think, but it’s hard to be very dogmatic or specific about how those examples turn into a blanket doctrine.
"I will go to him [in the grave/in the ground], but he will not return to me [from the grave/from the ground]."
You can't assume your interpretation about babies going to Heaven is accurate, when there's a perfectly straightforward way to read that comment from David in a way that doesn't suggest your interpretation at all.
God is not sweetness and light toward anyone, babies included. He killed a ton of babies-- during the flood and in various military campaigns he was the general of (to understate God's role) and at other times. He's not touchy-feely or a "sensitive New Age" guy.
In fact, in 2 Samuel, Nathan gives David the news that God is about to bring some four-fold major punishment to David, and it involves punishing others around him for the purpose of punishing David (like a sociopath who kidnaps an enemy's child and tortures them because it tortures the parent-- torturing the child is hardly a concern to him), including the rape of David's daughter Tamar. If God is punishing David by punishing his child, why would he give David a light sentence and give him an indication that the baby will be okay ultimately in heaven. God wants David to feel very bad about this punishment, and not have some silver lining to consider.