Permanent Digital Data Storage: Store it and forget it

Permanent Digital Data Storage: A Materials Approach. Barry Lunt, Robert Davis, Douglas Hansen, John Dredge, Hao Wang and Matthew Linford. International Conference on Preservation of Digital Objects.  iPres 2013.
      "For deep archival storage, the ideal is to be able to create the desired artifact, then store it and forget about it, knowing that whenever we wish to access it again, it will still be there, and we will still be able to read or observe it." Using ancient historical materials as examples, archival storage should be made from materials which either do not oxidize (such as gold), or else are by nature fully oxidized or chemically-reacted materials. 

Merely making a mark in a layer of recording material is not robust data storage solution. By using layers of inorganic, stable materials, a digital medium can be created for recording and playback. Optical data recording has a significant advantage when it comes to data longevity because accessing the object does not involve any contact between the media and the recording and playback mechanism. This means the file can be read an infinite number times without any degradation. "And the relative simplicity of the playback mechanism means that, if the data persists on the media, future optical playback systems will readily be capable of being adapted as necessary to read data stored permanently on optical discs".

A materials approach to the problem of permanent digital data storage shows that by using extremely durable materials they can create a permanent storage medium. Permanent, in this case, does not mean 'forever', since "nothing that science acknowledges is permanent". The paper uses the ANSI/AIIM definition of permanent: “Life Expectancy: Length of time that information is predicted to be retrievable in a system under extended-term storage conditions. Rating for the ‘life expectancy’ of recording materials and associated retrieval systems. NOTE: The number following the LE symbol is a prediction of the minimum life expectancy in years for which information can be retrieved without significant loss when properly stored under extended-term storage conditions, e.g., LE-100 indicates that information can be retrieved for at least 100 years of storage."

The authors propose that a practical definition for permanent digital data storage would be: "Permanence: The ability of a digital data storage medium to last at least two hundred years without significant deterioration under normal use and storage conditions in libraries and archives. This means there is a 99.99% confidence of complete data recovery using the intended read mechanism or hardware."

Currently the only digital medium that can last for more than a decade or so is the M-Disc, from Millenniata, Inc. All other digital data storage media have very serious limitations when it comes to permanence. How can the life expectancy of a medium be established? One way is using the long accepted methods of accelerated testing using the Arrhenius and Eyring equations as the scientific foundation.

The paper looks at solid-state storage media and computer tape and explains why they are not permanent. None of the current solid-state storage options are viable for permanent digital data storage. The problem with magnetic media is that "the magnetic domains begin to relax, slowly reverting to their original random orientation, and slowly degrading the difference between the encoded 1s and 0s. Eventually, so many of these bits will have degraded that reading a file back will have become impossible." Magnetic tape uses an organic material as a binder, which will degrade with time; the recording layer separates from the polyester substrate causing permanent data loss. There is an urgent need for some way to store digital data permanently. The M-Disc is one viable solution and "we are well on our way to providing two additional media for permanent digital data storage."

Just having a permanent storage medium is not a complete solution without a way to read the data far into the future. But a permanent medium is indispensable for deep archival data storage. "Some future generation may struggle to learn how our data was stored, but if all the marks on the Rosetta stone had faded away, deciphering them would never have been possible. It is relatively easy and inexpensive to read the marks made on optical discs - the hardware is widely available and optical discs are the most widely adopted digital storage medium in history. And wide adoption is a powerful predictor of relative permanence of readability."

Related posts:

• Permanent Digital Data Storage: Tape, solid-state, and discs
• So what's up with the M-Disc? Part 1   Part 2
• Fighting entropy and ISIL, one image at a time
• Preserving Our Digital Heritage
• Dataliths vs. the digital dark age

Digital Preservation Matters.