[St_Thomas_Aquinas]

Joseph and Mary had sexual relations and had other children.

The angel told her, "Don't be afraid, Mary. You have found favor[a] with God.
31 You will become pregnant, give birth to a son,
and name him Jesus.
32 He will be a great man
and will be called the Son of the Most High.
The Lord God will give him
the throne of his ancestor David.
33 Your son will be king of Jacob’s people forever,
and his kingdom will never end.”
34 Mary asked the angel, "How can this be? I'm a virgin."

The angel told Mary that she would become pregnant. Why would Mary object, since she was engaged to be married? And what would being a virgin have to do with anything?

Look through the writings of the early Christians. This has always been interpreted as contradicting her vow of perpetual virginity. Even Luther believed that Mary was ever-virgin.

[ealgeone]

According to the world-renowned patristics scholar, Johannes Quasten: "The principal aim of the whole writing [Protoevangelium of James] is to prove the perpetual and inviolate virginity of Mary before, in, and after the birth of Christ" (Patrology, 1:120–1).

http://www.catholic.com/tracts/mary-ever-virgin

The following is from wikipedia:

The document presents itself as written by James: "I, James, wrote this history in Jerusalem."[3] The purported author is thus James, the brother of Jesus, but scholars have established that the work was not written by the person to whom it is attributed.[4]

That conclusion is based on the style of the language and the fact that the author describes certain activities as contemporary Jewish customs that probably did not exist. For example, the work suggests there were consecrated temple virgins in Judaism, similar to the Vestal Virgins in pagan Rome, although this is unlikely to have been a practice in mainstream Judaism.[citation needed] Conversely, some Eastern Orthodox Christians and Roman Catholics argue that that the Old Testament shows that consecrated virginity had been practiced in Judaism since the days of the prophet Samuel (1 Samuel 2:22),[5] and the idea of Mary being a consecrated temple virgin is not far-fetched.

For context the quote from 1 Samuel 2:22

22 Now Eli was very old; and he heard all that his sons were doing to all Israel, and how they lay with the women who served at the doorway of the tent of meeting. I hope catholics aren't relying upon this to prove their point.

Not sure where catholics get virgins in this passage. Laying doesn't mean they were taking a nap!

That this was rejected by the early church is evidenced by it not being included in the canon.

If roman catholics really believed this to be the case, why wasn't it included in the canon at Trent??

For the catholic to continue to argue otherwise is to neglect the context of the NT on those passages that deal with Joseph and Mary having other children.