They would not have been Mary's children, since Jesus was concerned to put her into the care of His beloved disciple John after His death, unnecessary and unthinkable if she had grown children to look after her.
No one thought such a carnal-minded thing --- Mary bearing more children --- until 1500 years after Mary's passing from this life.
The early Christians would have died to defend Mary's perpetual virginity. You can see it written on the catacomb walls of the underground cemeteries under the city of Rome. Thus attested the martyrs: "Beatae Mariae Semper Virgine."
`
`
`
From the Book of Mark, Chapter 6
1) Jesus went out from there and came into His hometown; and His disciples….
2) When the Sabbath came, He began to teach in the synagogue; and the many listeners were astonished, saying, “Where did this man get these things, and what is this wisdom given to Him, and such miracles as these performed by His hands?
3) Is not this the carpenter, the son of Mary, and brother of James and Joses and Judas and Simon? Are not His sisters here with us?”
Those who cannot see this fact are the ones who insist on the perpetual virginity of Mary! This is contrary to Scripture