I think Pompey walked into the Temple in 63 (BC) and said the place was empty.
The Ark was never in the 2nd Temple, which is the one that Pompey would have seen. It is last mentioned circa 620 BC when King Josiah orders the Levites to return it to the Temple, but there is no mention of where the Levites had taken it or if they ever returned it. Then it disappears from history.