Mark refers to Jesus as the son of Mary. He dosesn't make that reference to any of the others Okay, but like the other discussions here have pointed out, silence on an issue doesn't mean much. None of the gospels talk about the middle 30 years of his life, but we know it happened.
I think that would be everybody's point. Canonical scripture doesn't address one way or the other how the brothers and sisters referred to in Mark were born. Those of us who believe in Mary's perpetual virginity do so by inference; those who believe that Mary had children subsequent to Jesus found their belief on inference as well. I think the inference in support perpetual virginity is stronger than the inference against it; you don't.