Of course -- the Church always taught that baptized infants who died went straight to Heaven. Definitely. It taught that Original Sin is a bar to heaven, but also that a just and merciful God would not consign an innocent to Hell. Hence, the hypothesis of Limbo -- not a doctrine because the Church didn't claim to know the what and how of the fate of dead unbaptized infants.
"Bapstism of blood" and "baptism of desire" ring a bell?
So where do unbaptized babies go?
If original sin is a bar to heaven and baptism removes that original sin, then the baby is not innocent until it is baptized, hence the fear of the parents.
By the church's own teaching, the baby is consigned to hell then because it is not innocent.