You hit the nail on the head. That's EXACTLY why the ruling elite in Israel won't even consider the possibility of the Temple actually belonging somewhere else.
Even worse... They have created an eternal conflict against Islam for it. I can understand the hate for Islam, but Islam was not even a religion yet when it was destroyed. The action of the Babylonians has been twisted into blaming Islam for it. They could at least be honest about this historical fact. They have a dozen other legitimate real reasons to hate Islam.