Just a matter of respect . . . and the absolutely unanimous opinion of the Church up to modern times (including Luther and various other Protestants as well).
If she had other children, I don't understand how that would make her any less sacred.
These are things I don't lose sleep over and one that shouldn't be a stumbling block between Catholics and Prots.