More from him (I didn't want a super long post):
It is unlikely that the term "brother" is being used figuratively or mystically because all Christians are Christ's brothers in that sense, making it pointless to single out certain individuals for this description. Full brother is impossible, as Protestants also acknowledge, since Jesus was not the biological child of Joseph. Half-brother is ruled out by the fact that Mary remained a virgin. It is possible they were adoptive brothers, but there does not seem to be any evidence for this in the biblical or patristic record.
More plausibly, they were step-brothers: children of Joseph who were Jesus' brothers by marriage. There is some evidence for this in the writings of early Christians. The earliest discussion of the matter that we have--in a document known as the Protoevangelium of James (c. A.D. 120)--states that Joseph was a widower who already had a family and thus was willing to become the guardian of a consecrated virgin. Though not inspired, the document was written within living memory of Mary, when Christ's family was still well known, as other sources attest (e.g., second century historian Hegisippus). It may contain accurate traditions regarding the family structure.
The step-brother hypothesis was the most common until St. Jerome (the turn of the fifth century), who popularized the idea that the brethren were cousins. One would not guess this from a casual reading of the New Testament, but many have tried to deduce it from statements in the New Testament.
Part of the issue turns on the meaning of the word "brother." Thus far we have been discussing the English word brother for simplicity. The Greek equivalent (adelphos) includes the same concepts in its range of meaning. But Greek also has a word for "cousin" (anepsios), which seems to have been the normal word used when referring to cousins. An advocate of the cousin hypothesis would need to explain why it wasn't used if Christ’s brethren were cousins.
The standard explanation is that the New Testament isn't ordinary Greek. Some have suggested that parts of it may be translations from Aramaic. It is unknown if or how much of the New Testament had an Aramaic original, but even if none did, Aramaic had a strong influence on it. Probably all the New Testament authors except Luke were native Aramaic-speakers, and much of the dialogue in the Gospels originally occurred in Aramaic. Sometimes the Gospels even tell us the original words (e.g., “Talitha cumi” in Mark 5:41).
This is important because the meaning of the Aramaic word for "brother" (aha) not only includes the meanings already mentioned but also includes other close relations, including cousins.
In fact, there was no word for "cousin" in Aramaic. If one wanted to refer to the cousin relationship, one has to use a circumlocution such as “the son of his uncle” (brona d-`ammeh). This often is too much trouble, so broader kinship terms are used that don’t mean “cousin” in particular; e.g., ahyana ("kinsman"), qariwa ("close relation"), or nasha ("relative"). One such term is aha, which literally means “brother” but is also frequently used in the sense of “relative, kinsman.”
The first Christians in Palestine, not having a word for cousin, would normally have referred to whatever cousins Jesus had with such a general term and, in translating their writing or speech into Greek, it is quite likely that the Aramaic word aha would have been rendered literally with the Greek word for brother (adelphos).
Excellent references there.
It's just as likely then that the Aramic speaking Jesus would have called his cousin an uncle, since the cousin was directly in the uncle's family...Unless of course, one wanted to convince people that Jesus had no brothers...Then you can call cousins, brothers...
"You say that Mary did not continue a virgin: I claim still more, that Joseph himself on account of Mary was a virgin, so that from a virgin wedlock a virgin son was born. For if as a holy man he does not come under the imputation of fornication, and it is nowhere written that he had another wife, but was the guardian of Mary whom he was supposed to have to wife rather than her husband, the conclusion is that he who was thought worthy to be called father of the Lord, remained a virgin."
Jerome - Against Helvidius
Where did the "cousin" theory develop?
In my, not so humble, opinion it makes no difference whatsoever whether Mary remained a virgin or not. In no way could it detract from her blessedness or the Divinity of Jesus.