A new way to stop digital decay
Sep 15th 2005
From The Economist print edition
From The Economist print edition
Computing: Could a “virtual computer”, built from software, help to save today's digital documents for historians of the future?
Posted on 09/20/2005 4:13:50 PM PDT by Zuben Elgenubi
A new way to stop digital decay
Sep 15th 2005
From The Economist print edition
Computing: Could a “virtual computer”, built from software, help to save today's digital documents for historians of the future?
WHEN future historians turn their attention to the early 21st century, electronic documents will be vital to their understanding of our times. Old web pages may not turn yellow and brittle like paper, but the digital documents of today's culture face a more serious threat: the disappearance of computers able to read them. Even a relatively simple electronic item, such as a picture, requires software to present it as a visible image, but 100 years from now, today's computers will have long since become obsolete. More complex items, like CD-ROMs or videos, will be unreadable even sooner.
In 1986, for example, 900 years after the Domesday book, the BBC launched a project to compile data about Britain, including maps, video and text. The results were recorded on laserdiscs that could only be read by a special system based around a BBC Micro home computer. But since the disks were unreadable on any other system, this pioneering example of multimedia was nearly lost for ever. It took two and a half years of patient work with one of the few surviving machines to move the data on to a modern PC (it can be seen online at www.domesday1986.com).
National libraries are just starting to grapple with this problem as part of their new mandate to preserve digital culture. “It is a major problem, but it is remarkable how little known it is,” says Hilde van Wijngaarden, head of digital preservation at the National Library of the Netherlands. “People just accept that things no longer work after ten years.”
Keeping working examples of all computer hardware is impractical, so the most popular preservation strategy is to copy files from one generation of hardware to the next. The problem is that today's word processors and web browsers, for example, do not always display files in the same way that older software did. An accumulation of subtle errors can eventually make the original item unreadable. An alternative approach, called emulation, uses software to simulate the old hardware on a modern computer, to allow old software to run. But today's emulators will need another emulator to run on the next generation of hardware, which will need another emulator for the next generation, and so on. This can also introduce errors.
So the National Library of the Netherlands is exploring a third option, using a simulated computer that exists only in software. It is called the Universal Virtual Computer (UVC) and is being developed by IBM, a computer giant. The researchers are writing programs to run on this virtual computer that decode different document formats. Future libraries will have to write software that emulates the virtual computer on each new generation of computer systems. But once that is done, they will be able to view all their stored documents using the decoders written for the virtual computer, which only have to be written once. “The decoder can be tested for correctness today, while the format is still readable,” says Raymond van Diessen of IBM.
His team has written decoders for two common image formats, JPEG and GIF. They plan to move on to Adobe's PDF format. IBM is also talking to drug firms, which are required to store data from clinical trials for long periods. Ultimately, the aim is to be able to preserve anything from simple web pages to complex data sets. Ominously, some scientific data from the 1970s has already crumbled into unreadable digital bits.
How hard would it be to construct a floppy drive with a voice-coil actuator for the heads and a two-channel 10Mhz ADC data capture front-end? I would think that three such drives (one each for 8", 5.25", and 3.5") would be able to read 99.99% of the floppies produced in those sizes (and would also, with proper programming, be better able to deal with bit rot than the drives of yesteryear.
Good idea, supercat. Apply for the patent tomorrow.
More like "Sky Is Falling" ping. (no offense intended:)
Information (data) has been 'lost' thru eons of history, but somehow scientists and historians have been able to recreate a fairly accurate picture of the past using only some of the smallest parcels of indirct data still in existance ... everything from the geological record, to the development of biology, to the origins of the Universe as we know it, came from these indirect scientific observations. It is doubtful that mere computer data will simply 'disappear' so easily as long as mankind is around to maintain it. There will always be someone around who still has a Timex/Sinclair antique PC sitting in their workshop :)
I managed to recover about a hundred 20 MB Bernoulli cartridges one time. The OS ain't the problem.
Actually, one of the biggest problems, IMHO, is going to be determining what material is worth moving to newer formats. Even if it only takes ten seconds to convert the contents of a floppy into modern format, what is somebody with a bunch of floppies and no particular clue what's one them supposed to do with them? One of the advantages of printed material is that in many cases one can pick up an item, look at it, and have some clue what it is. Even with movie film that can be somewhat possible if one has good eyesight (much easier with 35mm than 8mm, though!). Another difficulty--and this applies not just to computers but to all types of material--is the loss of metadata. A computer may be able to tell that a file contains a picture, and a human may be able to tell that it contains a picture of a woman holding a baby. But who are the woman and the baby? If there isn't anyone around to identify the significance of a picture, that significance will be lost even if the picture itself remains.
"My name is Ozymandius, king of kings,"
"Look on my beebers, ye mighty, and be stuned."
PING
Some problems are just too easy.
ping for later.
-PJ
> ... one of the biggest problems, IMHO, is going to be
> determining what material is worth moving to newer formats.
And then you'll discover that the worthwhile stuff is
all DRM-protected and unreadable. It may even be out of
copyright, but the lawyers who hold the decryption keys
will, alas and joy, be long dead.
ping
That's why my office only uses microfilm to preserve our old documents. I am a records manager and oversee the microfilming operation in one office in our county courthouse. They tell me that the newer microfilm has a shelf-life of 500 years if stored properly. The technology to read microfilm is so simple and will always be available. We have considered scanning/microfilming, but the equipment is very expensive. I'm hoping that we will be able to do that eventually - it would certainly speed up the process!
We have to be able to dispose of records but I would refuse to throw them out if they were on disk-only. Heck, I have computer games from five years ago that I can't play on my newer computer!
...There will always be someone around who still has a Timex/Sinclair antique PC sitting in their workshop :)...
or two...
with 16k RAM modules and audio cassette software
// They tell me that the newer microfilm has a shelf-life of 500 years if stored properly. //
Unfortunately, a lot of stuff on microfilm is barely legible even when new. As for long-lived storage, how about daguerotype? That's stored as metalic mercury on metalic silver, right?
Are those the original bottle caps used to prop up the 16k RAM module, or did you eventually solder and/or hardwire the pack to the back of the unit ? /grin
Many years ago, American Heritage magazine printed a circa-1890s photo of a man, a woman, and a horse. They dutifully noted that the only information about the photo was from a handwritten caption on the back, "The horse's name is Fred."
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.