And they seem to have deserved it at the time because they thought he was out of his mind.
In his epistle, James identifies himself as a "servant" of Christ, Jude identifies himself the same way and adds that he is the brother of James. No person in the Bible identifies themselves as a sibling of Jesus, nor is anyone identified as a child of Mary aside from Jesus.
When he write official correspondence in Florida, do you think Jeb Bush identifies himself as the President's brother? Our Lord's brothers were not quick to believe upon Him and even tried to give Him unsound advice. I wouldn't want to call them knuckleheads, but the eleven disciples didn't need to take a second seat to any of them.