First, following Jesus in his adult ministry did not entitle Matthew to any particular insight into Mary's precise relation to the "brothers" of Jesus. Second, it was simply not Matthew's focus, in either of the passages, to convey either knowledge or speculation that he might have had.
I'm only talking from a common sense POV. Have you ever had a roommate? These people traveled together, ate together, slept together, prayed together etc. They did all this and yet you are saying that they didn't share with each other, EVER?! Don't Christians share with each other? We can debate whether Matthew was purposefully revealing it, but I still believe he had to know one way or the other.