First, whether the witnesses saw actual plates, or only had visions of the plates, it is apparent that the plates where not necessary in the translation of the Book of Mormon. In Whitmers address, he describes the translation process:
I will now give you a description of the manner in which the Book of Mormon was translated. Joseph Smith would put the seer stone into a hat, and put his face in the hat, drawing it closely around his face to exclude the light; and in the darkness the spiritual light would shine. A piece of something resembling parchment would appear, and on that appeared the writing. One character at a time would appear, and under it was the interpretation in English. Brother Joseph would read off the English to Oliver Cowdery, who was his principal scribe, and when it was written down and repeated to Brother Joseph to see if it was correct, then it would disappear, and another character with the interpretation would appear. Thus the Book of Mormon was translated by the gift and power of God, and not by any power of man.
So by the accounts of the witnesses, the plates remained covered during the translation. The Book of Mormon came about because Joseph Smith stuck a rock in his hat, followed it with his face, and spoke the translation for Cowdery to write down. How does the existence of plates fit into this scenario? It does not.
If you lived in the 19th century and you were to translate an entirely foreign language, only possessing a modicum of formal schooling, and you were to rely almost entirely upon the whisperings of the Holy Ghost and unfamiliar tools of divine origin, and it required the utmost concentration and zero distraction and you wanted to get it exactly right, how would you do it? I'm sure that your method would earn the scorn of others too.
Look, if our Heavenly Father wanted this to be easy, He and His son, Jesus Christ, would have appeared to the entire world and would have revealed all of the records that He commanded His prophets to record throughout the ages. But what faith would that require? And if we are not to be proved by our faith, for what purpose do we owe this mortal existence?
Without them, it comes across - with all due respect - to the non-LDS person as just made up.