No traditional Jewish religious authority would accept any location but the exact site of the First and Second Temples. Period.
Since non-Jews would not be allowed on the Temple Mount, what would be the point of leaving Muslim shrines there?
The younger “traditional” students have been arguing otherwise. There already is some questioning otherwise as to the “exact” location of the previous temples.One could build it adjacent to the wailing wall some have argued.