If there is an error, the error would be in the average as well.
For whatever reason, the polls seem to get much more accurate right before the election.
With the early voting and vote by mail, I suspect when you get close enough to the election, you are almost already getting an exit poll.
I agree. Also, there are fewer undecideds right before election day, which can increase the accuracy a bit.