Where in scripture does it say he wouldn't?
It doesn't. So if I were a sola scriptura Protestant, I really wouldn't be able to comment on the matter of Mary's perpetual virginity, one way or the other.
Where in scripture does it say he wouldn't?
1 Corinthians 7:5John Paul II puts St. Paul's remark into context:
"Do not deprive each other except by mutual consent and for a time, so that you may devote yourselves to prayer."
"In Egypt there was a community of women who, associated with the Essene spirituality, observed continence. These women, the Therapeutae, belonging to a sect described by Philo of Alexandria (De Vita Contemplativa, 21-90), were dedicated to contemplation and sought wisdom.
It does not seem that Mary ever knew about these Jewish religious groups which practiced the ideal of celibacy and virginity. But the fact that John the Baptist probably lived a celibate life and that in the community of his disciples it was held in high esteem would support the supposition that Mary's choice of virginity belonged to this new cultural and religious context.
...Thus it should be maintained that Mary was guided to the ideal of virginity by an exceptional inspiration of that same Holy Spirit who, in the course of the Church's history, will spur many women to the way of virginal consecration.
The singular presence of grace in Mary's life leads to the conclusion that the young girl was committed to virginity. Filled with the Lord's exceptional gifts from the beginning of her life, she was oriented to a total gift of self--body and soul--to God, in the offering of herself as a virgin.
In addition, her aspiration to the virginal life was in harmony with that "poverty" before God which the Old Testament holds in high esteem. Fully committing herself to this path, Mary also gives up motherhood, woman's personal treasure, so deeply appreciated in Israel. Thus she "stands out among the poor and humble of the Lord, who confidently hope for and receive salvation from him" (Lumen gentium, n. 55). However, presenting herself to God as poor and aiming only at spiritual fruitfulness, the fruit of divine love, at the moment of the Annunciation, Mary discovers that the Lord has transformed her poverty into riches: she will be the Virgin Mother of the Son of the Most High. Later she will also discover that her motherhood is destined to extend to all men, whom the Son came to save (cf. Catechism of the Catholic Church, n. 501)."
Our Lady Intended to Remain a Virgin
Pope John Paul II
General Audience, July 24, 1996