It’s just you. Robert Barnes, John Eastman, and many others have argued the presiding officer of the Senate (see senate rules) can entertain objections from the floor at any time.
For one thing, I would be inclined not to give much credence to the input from many lawyers on these matters.
The problem is that a lawyer is an advocate for his client and/or their political agenda, so he is prone to take outlandish positions just because that's what he's paid to do. That's how you end up with some hack like Mark Levin (for example) getting on the airwaves and taking diametrically opposed positions about the VP's authority in the 2000 and 2020 elections.
Secondly, there is a huge distinction between "entertaining objections" and "counting whatever he wants" when it comes to the VP's role as the President of the Senate.
In a presidential election, the role of the states is a plenary role -- which means even Congress cannot override the will of a state when the electoral votes are cast. That doesn't give Congress much leeway at all. In fact, I would suggest that any power given to Congress to reject electors would completely undermine the whole constitutional principle of an independently elected Executive Branch of government.