And that is what's going to happen tomorrow. But Pence can adjourn the meeting to complete its business on a later day if he finds that some electoral votes are untrustworthy. That should still meet the letter of the law.
But what gives him the authority to decide electoral votes are untrustworthy? Isn't that what the challenges are for? The law gives the power over those to the House and Senate.