I'm not aware of anyone claiming these to be brothers of Jesus.
The context indicates they are not.
Much as Paul uses ἀδελφοί in Galatians 1:2 to identify fellow believers. No one believes these are his actual brothers. Why? Context.
Matt 27:56 identifies "Mary the mother of James and Joseph" as one of the women who ministered to Jesus, not as His mother. She is evidently the wife of Clopas.
Perhaps you're aware Mary, James and Joseph were fairly common names back in the day.
Jesus did, too, to mean somebody close to you (as in Matt. 5:22-24, 7:3-5, 12:50,18:15, and many other places in the NT.
And again...context, which I'm beginning to believe is a new concept to the Catholic, tells us how to understand "brother" in these verses.
However, these are not the verses in question.
These are the verses we're discussing.
The context of each tells us Jesus had brothers and sisters and they came from Joseph and Mary. Any other understanding defies common reading.
While He was still speaking to the crowds, behold, His mother and brothers were standing outside, seeking to speak to Him. Matt 12:46
"Is not this the carpenter's son? Is not His mother called Mary, and His brothers, James and Joseph and Simon and Judas? Matt 13:55
And His sisters, are they not all with us? Where then did this man get all these things?" Matt 13:56
"Is not this the carpenter, the son of Mary, and brother of James and Joses and Judas and Simon? Are not His sisters here with us?" And they took offense at Him. Mark 6:3
And His mother and brothers came to Him, and they were unable to get to Him because of the crowd. Luke 8:19
And it was reported to Him, "Your mother and Your brothers are standing outside, wishing to see You." Luke 8:20
After this He went down to Capernaum, He and His mother and His brothers and His disciples; and they stayed there a few days John 2:12
Therefore His brothers said to Him, "Leave here and go into Judea, so that Your disciples also may see Your works which You are doing. John 7:3
For not even His brothers were believing in Him. John 7:5
Do we not have a right to take along a believing wife, even as the rest of the apostles and the brothers of the Lord and Cephas? 1 Corinthians 9:5
But I did not see any other of the apostles except James, the Lord's brother. Galatians 1:19
35 For I have come to turn ‘A man against his father, a daughter against her mother, a daughter-in-law against her mother-in-law. 36 A man’s enemies will be the members of his own household.’ 37 Anyone who loves his father or mother more than Me is not worthy of Me; anyone who loves his son or daughter more than Me is not worthy of Me;
Airing some dirty laundry here?