Perhaps, but then there would be a large number of loose ballots laying around. That would be another problem.
Loose ballots are a minor problem compared to no ballots.