So you're suggesting that someone other than Mary was his mother??
What He says in that passage is that all who keep His word are His brethren, and Mary's children. He repeated that point more forcefully on the Cross. "Behold, your son". This made us all her children:
the dragon was angry against the woman: and went to make war with the rest of her seed, who keep the commandments of God, and have the testimony of Jesus Christ.(Apoc. 12:17)