Add a Comment to this Note (list members only)
Dead medium: Dead Digital Documents
From: sblack@library.berkeley.edu (Steven Black)

Source(s): Business Week magazine, April 20, 1998

(((Steven Black remarks: The more we think we save, the more is actually lost!)))

(((bruces remarks: here's yet another set of colorful, horrific anecdotes on the fragility of digital storage media, this time appearing in the mainstream business press. It seems very likely that this once-arcane problem will slowly intensify into open scandal.)))

"DATA STORAGE: FROM DIGITS TO DUST

"Surprise == computerized data can decay before you know it

By Marcia Stepanek in New York

"Up to 20% of the information carefully collected on Jet Propulsion Laboratory computers during NASA's 1976 Viking mission to Mars has been lost. Some POW and MIA records and casualty counts from the Vietnam War, stored on Defense Dept. computers, can no longer be read. And at Pennsylvania State University, all but 14 of some 3,000 computer files containing student records and school history are no longer accessible because of missing or outmoded software.

(...)

"The Information Age is creating a digital dilemma. For years, computer scientists told us that digital 1s and 0s could last forever. But now, we're discovering that the media we're using to carry our precious information on into the future are turning out to be far from eternal == so fragile, in fact, that some might not last through the decade. More is at risk than government and corporate records. The danger extends to cultural legacies: new music, early drafts of literature, and academic works originate in digital form == without hard copies.

(...)

"'Digital information lasts forever, or five years == whichever comes first,' says Jeff Rothenberg, senior computer scientist at RAND Corp.

"Forget forever. Under less-than-optimal storage conditions, digital tapes and disks, including CD-ROMs and optical drives, might deteriorate about as fast as newsprint == in 5 to 10 years. Tests by the National Media Lab, a St. Paul (Minn.)-based government and industry consortium, show that tapes might preserve data for a decade, depending on storage conditions. Disks == whether CD-ROMs used for games or the type used by some companies to store pension plans == may become unreadable in five years.

"For consumers, the biggest worry is CD-ROMs. Unlike paper records, CD-ROMs often don't show decay until it's too late. Experts are just beginning to realize that stray magnetic fields, oxidation, humidity, and material decay can quickly erase the information stored on them.

"Says Robert Stein, founder of New York-based Voyager Co., which makes commercial CD-ROM books and games: 'CDs have a tendency to degrade much faster than anybody, at least in the companies that make them, is willing to predict.' Stein doesn't expect the CD-ROMs Voyager sells to last more than 5 or 10 years, and neither, he says, should customers.

"There's another problem: the unrelenting pace of technology. Chances are good that the software needed to get at much of today's data might not be readily available in 10 years. Anyone who has tried wrestling information from a 5 1/4-inch floppy disk knows that. Just ask scientists conducting rain forest research. Satellite photos of the Amazon Basin taken in the 1970s == data critical to establishing deforestation trends == are trapped on indecipherable magnetic tapes no longer on the market.

"But even keeping a step ahead of data decay and software obsolescence is no guarantee of escaping the problem. Companies spending heavily on sophisticated new computers and software to beat the technology reaper say they're beginning to run into a whole new problem. All too often, when they transfer information from one aging media or computer system to a newer one, not all bits make the migration.

"Sometimes, just a footnote or spreadsheet is lost. Other times, whole categories of data evaporate. Says Rothenberg: 'It's like playing the child's game of Telephone. It doesn't take many translations from one media to another before you have lost significant aspects of the original data.'

"The Food & Drug Administration reports that some pharmaceutical companies are discovering errors as they copy drug-testing data that back up claims of long-term product safety and effectiveness. In several recent cases involving data transfers from Unix computers to systems running Microsoft's Windows NT operating system, blood- pressure numbers were randomly off by up to eight digits from those in original records, FDA and company data specialists report.

"Sophisticated software can catch most of the errors, but 'not all the time,' says Rone Lewis, vice-president of business development of Surety Technologies, a data recovery and migration firm.

"Some companies fear the problem could expose them to lawsuits. 'In our litigation-prone age, it's harder to defend yourself if you're losing parts of your records when you migrate them,' says Henry Perritt, dean of Chicago Kent College of Law.

(...)

"Ray Paddock, a director for Storage Technology Corp., says the problem is so bad for some of his clients that they're creating new databases just to decipher the data they have on tape and disks. Others, he says, are simply keeping the old version of the software used to create documents.

"NO STANDARDS. Meanwhile, the government is looking into establishing durability standards for digital media. A task force == including representatives of Eastman Kodak, IBM, and archivists at leading museums and universities == has agreed on a digital longevity test ultimately aimed at increasing the life span of CD-ROMs and other types of digital media. The only problem: So far, no manufacturer has tested its products using the age-test created by the task force. And the group is still working on a standard for magnetic tape.

"Others are at work on new technologies to solve the problem. NORSAM Technologies in Los Alamos, N.M., for example, is promoting its HD-Rosetta project, which permanently stores historical documents == but only if they are converted from digital back to analog recording formats.

"But at least one remedy being offered by researchers sounds a lot more like the distant past than the future: Cobblestone Software Inc. in Lexington, Mass., is promoting PaperDisk, which uses paper to print out complex patterns of dots and dashes representing digitized files. Cobblestone President Tom Antognini claims it should last for centuries == or about as long as old-fashioned, high- quality paper."

Steven Black (sblack@library.berkeley.edu)