"Then why don't we just let the silence of the scripture remain silent...Since that is all that God has told us, shouldn't that be enough."
Because:
a) Scripture isn't silent - in its typology it is deafeningly loud
b) Even if it were totally silent, that would not affect the veracity of the Tradition for:
"There are also many other things that Jesus did, but if these were to be described individually, I do not think the whole world would contain the books that would be written." Jn 21,25.
c) This is what Christians have always and everywhere believed. Therefore, any contradictory view is novel doctrine and falls under anathema.
d) Christ's Church has never subscribed to a doctrine of Scripture alone, but has always held fast to what has been handed on by both Scripture and Tradition. The Church's prayer and divine liturgy existed before any of the New Testament was committed to writing, for example.
e) Scripture is completely silent about the destruction of the temple in 70 A.D. and yet you would consider me mad if I suggested we therefore believe it to be still standing, as it is a fact that the temple was so destroyed. The Church's doctrine about the Mother of God is similarly factual and therefore cannot be denied without denying Truth Himself.
The destruction of the temple in AD 70 was spoken of many times in prophecy. But your reasoning is in even greater shambles. I do not argue that silence (if it existed) concerning the temple should be used to prove that it still remains. I argue that silence (if it existed) concerning the temple does not allow me to formulate my own opinion concerning the temple and teach it as the word of God. I understand that catholics do not base their beliefs on scripture, but if they are confident in this aspect, neither should they attempt word games to make scripture support their already held doctrine.
Actually, Christ predicted it--although if you meant that the Bible doesn't record the event itself, you're correct.