"You say that Mary did not continue a virgin: I claim still more, that Joseph himself on account of Mary was a virgin, so that from a virgin wedlock a virgin son was born. For if as a holy man he does not come under the imputation of fornication, and it is nowhere written that he had another wife, but was the guardian of Mary whom he was supposed to have to wife rather than her husband, the conclusion is that he who was thought worthy to be called father of the Lord, remained a virgin."
Jerome - Against Helvidius
Where did the "cousin" theory develop?
In my, not so humble, opinion it makes no difference whatsoever whether Mary remained a virgin or not. In no way could it detract from her blessedness or the Divinity of Jesus.
For all of the talk of the Lord's "brothers and sisters" remember these things:
1. There is absolutely NOTHING in scripture that suggests that the Blessed Virgin Mary had any other children.
2. On the Cross, Jesus entrusted His mother to the apostle John. Had Mary had other children, this would have been a sinful violation of Judaic law. Christians are in complete agreement that the Lord NEVER sinned.
3. Because there is no established teaching on Joseph, it is quite possible that James and the others were his children from a prior marriage, in this case it would have been perfectly acceptable for Christ to entrust His mother to someone else. It is further possible that they were close relatives who were orphaned and raised by the Holy Family as if they were their own, in this case it would not be at all unusual for them to be referred to as the Lord's brethren.
4. Neither James nor Jude make any claim in their epistles that they or anyone else is a sibling of Jesus.
5. The perpetual virginity of the Blessed Mother WAS NEVER questioned until centuries AFTER THE START OF THE PROTESTANT REFORMATION. If you want to criticize the traditions of the Church, go ahead, but do so knowing that the idea that Mary bore other children is a VERY NEW tradition.
6. Now, I am aware of the argument that for Mary and Joseph to not have sexual relations would be "abnormal." However, the Holy Family was the most decidedly "abnormal" family to have ever lived. When two people have told an angel of God that they are willing to raise the Son of God, it is axiomatic that any expectation of a "normal" life has been forfeited.
7. And finally think about this, had the the Blessed Virgin Mary had other children, it would have created the very real and awkward possibility of a "royal lineage" of direct descendants of the Lord's natural brothers and sisters.