Depending on State law, envelopes (Outside) are registered and kept for a certain time period. If an audit uncovers a large percentage of spoiled ballots/law breaking i.e. preventing poll watchers from the count... a judge could technically rule in favor of the party harmed (I think this happened in a Pennsylvania State Senate election back in 1994). The kicker I believe was that the ruling came from a FEDERAL judge as well.
I don’t think you can match the envelope to the voter (because then it is no longer a secret ballot); do you want a Biden administration to have access to lists of people who voted for his opponent?