By my count, there have been Three Temples, one built by Solomon, one built after the Babylonian exile and one by Herod the Great who tore down the Second Temple completelty and built the Third. I have no idea why people say only Two Temples have been in existence. There was also the mysterious Temple in Elephantine which was destroyed by the Egyptians.
My understanding is that Herod added to the existing Temple complex without tearing it down, and that Temple services continued uninterrupted during that process.
Solomon’s Temple was rebuilt, not a second one built in place, so, you made an error
remember, they rebuilt the walls of Solomon’s Temple when they returned, they didn’t build a new one.