Well according to scripture the 'ark' was not left behind. So it was not in Herod's temple.
According to Chronicles, the ark was not among the temple treasures taken by Nebuchadnezzar to Babylon. It was apparently hidden before that time. There is no mention of the ark in any historical record after that point with the exception of the claim by the Ethiopian Orthodox Church to possess the ark.