Every election irregularity must be challenged.
Absolutely. And the only way to prevent double-voting is to keep good voter logs, and check absentee ballots against those.
And it must be done (1) after the polls close, and (2) before the ballot is opened. Unless it can be done automated, (by a code on the label) each absentee ballot must be individually handled looked at by a person.
I voted in the town hall early, I received a ballot with a printed sticker on it, with my name, etc. I'd guess it could be scanned, and the voter log marked as "voted". If it showed I'd already voted in person, then it could be rejected. Those that were problematic or unreadable would be rejected for hand-processing. That would allow the thing to be done relatively quickly.
I work a poll when I was a kid. You got there early, 5 or 6 o'clock, and didn't leave until 11pm, if things went well. Even as a young man, I wiped out by the time we wrapped up. They should have a couple of shifts so they don't have to stop the counting process because the poll workers are exhausted.