[Mark17]

Why weren’t the brothers and sisters of Jesus at the cross? We can only speculate. My speculation, for whatever it is worth. Maybe they didn’t want to see all the blood and gore. I think it’s possible that none of them were true believers yet. At least 2 became believers later, James and Jude, and I hope all of them did. Maybe more likely, Jesus’ brothers and sisters, were afraid the Jewish leaders would have them crucified as well.
We will never know what their thinking was.

[MHGinTN]

It is only my opinion but here goes.

It is possible that Joseph died leaving Mary with a step ladder of ages children, boys and girls. If there were little ones the Jewish custom was for aunts and uncles to take in the older boys and girls (they made good servants to the household don'tchaknow) and the little ones remained with their Mother. we have no indication that Mary remarried so it is possible that she was supported by carpentry work (we know Who was an apprentice to His carpenter father) and help from aunts and uncles. There is some indication that Mary had at least one wealthy close relative who owned lead mines.

We have the scene of Mary and the brothers and sisters (some or all?) showing up at a house seeking a private audience with Jesus, just as the division of Judaism was heating up and the Sanhedrin was getting their gumption up to arrest Jesus. So the family was not scattered afar if they show up to seek a private audience.