Sounds bizzare.
Well, we don’t vote for president, we vote for electors who are promised to a particular candidate. For a primary election, I think that a state court could rule on eligibility to be on the ballot, but most states have specific rules on who can be a candidate and most do not include anything about being a natural born citizen, only being a US citizen, and it’s even questionable if a state could disallow someone who has citizenship.
But ultimately, yeah, the only body that has the power to make this decision is the electoral college prior to the election, and after the election, impeachment could be used.