I don't think it would be possible for me to disagree with you more strongly. The Lord's teaching regarding the Tabernacle in the wilderness and the building and rebuilding of the Temple shows that he cares a very great deal about the places dedicated to his true worship.
If God cared that much about the Temple it wouldn't need rebuilding. It could never have been destroyed in the first place. These buildings mean something to only to us and they are used by God to communicate with us, nothing more. Christ did most of his teachings outside of the temples, Abraham did not worship in a temple, and Moses never got the chance to see the temple. They are symbols to US because we are small and think we need a bridge to reach God. There is nothing wrong with that, it's even very useful. But to tie your faith to a building, allowing your faith in God to be shaken because someone is playing loud music outside is downright silly.
It's interesting to note that the Palestinians rushed into a Christian church in order to receive sanctuary from the very "infidels" they are so opposed to. The simple fact is, they are children of God also. They do have to pay for the consequences of their actions, but sanctuary is not the action they have to pay for. It's the murder and the hatred and the lack of respect for human life. Human life, THAT is God's cathedral.