Matt. 13:53 And it came to pass, that when Jesus had finished these parables, he departed thence.
54 And when he was come into his own country, he taught them in their synagogue, insomuch that they were astonished, and said, Whence hath this man this wisdom, and these mighty works?
55 Is not this the carpenter's son? is not his mother called Mary? and his brethren, James, and Joses, and Simon, and Judas?
56 And his sisters, are they not all with us? Whence then hath this man all these things?
The statement is clear. The officers of the synagogue, no friend of Jesus and yet familiar with His family, familiar enough to identify the work his father does, his mother's name and his brother's name and that thefamily lives in the same community. They recognize Jesus as;
the carpenter's son,
Mary's son,
brother of James, Joseph, Judas and Simon,
brother of his sisters.
Then they say that the family "all" is still with them i.e. living among them. An outsider would not use the familiar term "brethren" to refer to Jesus' entourage.
Still this passage is equally consistent with the officers of the synagogue referring to Christ's cousins, once we realize that the word adelphos applied to cousins, and even more distant relatives. All the more so because James and Joseph are identified in Mark 15 as having different mother.
Your method is common to the Protestant analysis in general: take a short passage, use the vernacular translation literally, do not look at context, do not look at the related passages elsewhere, and thump the table with it. Do it when that contradicts the traidtional teaching. Do not do it when the same method confirms the traditional teaching (re-read James 2:24 through the same lense and get back to me). No sale.
Scripture does not say that these were born of Mary, the Mother of Jesus.