Well, unless Jesus' siblings were concieved miraculously, Mary could not have remained a virgin.
The verse doesn't say they were Mary's children. It says they were Jesus' brothers. To the Hebrews, all of your male contemporary (and even not very contemporary) relatives were your "brothers". That's why Abraham calls Lot his "brother" in Genesis, for example. Other verses make it clear that Lot was really Abraham's nephew.