It doesn’t take a challenge - some of the states sent two sets of electors and the President of the senate gets to choose how to vote.
No states sent two sets of electors. Only the legislature could approve the second slate and none of the legislatures did. So you have a slate selected in accordance with state law and a slate that just proclaimed themselves to be electors. Unless that second set gets a legislative seal of approval Congress won't deal with them.