I thought there were disagreements as to where the holy of holies had been placed, so it was safer to avoid going to the Temple Mount altogether. Mind you, I still place 99.99% of the fault with the Muslims, but I would feel better if the Jews were united defending their right to that holy place.
Perhaps there’s disagreement on its boundaries, but as far as I know, everyone agrees it was always atop that big ol’ rock.