The Holy of Holies was visible to the people from the Mount of Olives through the EASTERN GATE (Golden Gate) if all the gates in the temple were opened. The ONLY PLACE THAT THIS COULD BE SEEN IS IF THE TEMPLE WAS BUILT ON TOP OF WHAT IS NOW CALLED THE DOME OF THE SPIRITS WHICH IS SOUTH OF THE DOME OF THE ROCK, which means that both Dome of the Rock and the Temple can exist at the same time, which the Bible, speaking of the Tribulation Temple, refers to as the Court of the Gentiles.
Looks like you and I are actually in agreement concerning where the Temple itself is located. Best I can tell there are a lot of folks out there who think the Temple encompassed the entire court Herod built.
I wonder what did happen to that stone? Is it possible the Romans broke it up?