This is a popular misconception. By granting an annulment, the Church is not saying that there never was a legal marriage, which is the social contsruct by which children are granted legitimacy. That's why, when there is a legal secular divorce, the kids are not considered 'illegitimate' because they were born when that marriage was still intact.
An annulment simply states that the SACRAMENT received by the couple during the Catholic Wedding Ceremony was not valid for whatever reason. If the Sacrament is not valid, it is not a valid religious union in the eyes of the Church, regardless of the legal union conferred by the license granted by the State.
So children born to parents whose Sacramental union is later annuled are no different from those of any secular couple who later got divorced.
The reason the Church does not allow re-marriage after a divorce without first having been granted an annulment is because you can't receive that Sacrament twice, validly.
Thanks for the explanation. "What God has joined let no man put asunder." People seem so shocked when the Church takes its words seriously.