The (information) machine stops

The (information) machine stops. Gary McGath. Mad File Format Science Blog. March 14, 2016.
     The “Digital Dark Age” discussion comes up again.  Instead of asking what could trigger a Digital Dark Age, we ought to ask

1. what conditions are necessary and sufficient for the really long-term preservation of information,
2. what will minimize the risk of widespread loss of today’s history, literature, and news?

Our storage ability has increased but the durability of that storage has decreased. We deal with obsolescence and format, file, and device failures. "Anything we put on a disk today will almost certainly be unusable by 2050. The year 3016 just seems unimaginably far. Yet we still have records today from 1016, 16, and even 984 B.C.E. How can our records of today last a thousand years?"

The current practices rely on curation, migration, and hoping that storage providers will be around forever. Or that some institutions will take up the task of preservation and continue it forever. This requires "an unbroken chain of human activity to keep information alive". History shows that information is often neglected or destroyed, and in reality, only a tiny fraction has survived. "Today’s leading forms of digital storage simply can’t survive that degree of neglect." Abby Smith Rumsey writes, "The new paradigm of memory is more like growing a garden. Everything that we entrust to digital code needs regular tending, refreshing, and periodic migration to make sure that it is still alive, whether we intend to use it in a year, a hundred years, or maybe never." It is not a safe assumption that "things will always be the way they are today, maybe with some gradual improvement or decline, but nothing that will seriously disrupt the way we and future generations live."

However, we have to remember that people and information have survived many types of catastrophes. The original question in the post was "If an uninterrupted succession of custodians isn’t the best way to keep history alive, what is? The answer must be something that’s resilient in the face of interruptions." An important part of this is to avoid reliance on fragile protection; the keys are durability and decentralization. "The hard parts are avoiding physical degradation, hardware obsolescence, and format obsolescence. Physical durability isn’t out of reach. Devices like the M-disc have impressive durability."

"The way to address obsolescence is with designs simple enough that they can be reconstructed." We need decentralized archives in many places with different approaches. "The problem is solvable. The mistake is thinking that an indefinite chain of short-term solutions can add up to a long-term solution."

Related posts:

• Dataliths vs. the digital dark age

Digital Preservation Matters.

MDISC Archive Service

MDISC Archive Service. Website. Millenniata. January 29, 2016.
    The MDISC Archival Beta tool is now available. It automatically preserves all your photos and videos, past, present, and future by engraving them on M-Discs. The service archives files that are in a designated internet service, currently Google Photo, and archives the files to M-Discs, which is then delivered to the owner. [This is a service that I have been trying out for a few months - Chris.]

The free limited-time beta service gives you three months to try out unlimited file archiving on M-Discs. "Ultimate peace-of-mind comes when you can hold your data in your hands.  Your photos can't be lost, corrupted, hacked, or erased, and they'll last forever."

Included on the website is a video that shows people opening digital files they had saved for 8 years or more. One-third of Americans have lost photos and video and don't even know it yet. The automated archival service makes it easier to archive the files.

Digital Preservation Matters.

5 storage technologies I'm thankful for

5 storage technologies I'm thankful for.  Robin Harris. Storage Bits. November 27, 2015.
     The "basis of any civilization is the storage of its culture." Previous the culture was stored by physical means, books, art and such. As this changes to digital means, there are several storage technologies that the article mentions will help preserve culture long term:

• Data encryption. "Because digital data is easy to copy and share, we need encryption to keep what is ours, ours alone."
• The thousand year disc. The M-disc is the "only digital media with a lifespan as good as a well produced "book.
• Scale-out object storage. Files that can be easily accessed from multiple servers, such as cloud services.
• Advanced archive storage. "As we collect and store more information, archiving - not backup - becomes the critical success factor."
• Solid-state storage. This "has revolutionized mobile device and enterprise storage."

Digital Preservation Matters.

Digital Curation Decision Form

Digital Curation Decision Form. Chris Erickson. Harold B. Lee Library. November 13, 2015.
Latest version is found here: Policies and Procedures
     This is the [former] version of our Digital Curation Decision Form (old version). The form is used by subject specialists (curators, subject librarians, or faculty responsible for collections) to determine

• what materials should be included in our Rosetta Digital Archive; 
• whether additional copies are needed, including copies on M-Discs; and 
• whether or not the digital collection is a preservation priority. 

Additional questions ask about access to the preservation copies; the preservation actions needed; and directions on content options if format migration is needed. The form was created to help subject specialists determine what should be preserved, even if they are unaware of digital preservation topics. In practice, we complete the form during an interview with new subject specialists. Documentation will be added when the final version is approved.

Digital Preservation Matters.

Permanent Digital Data Storage: Store it and forget it

Permanent Digital Data Storage: A Materials Approach. Barry Lunt, Robert Davis, Douglas Hansen, John Dredge, Hao Wang and Matthew Linford. International Conference on Preservation of Digital Objects.  iPres 2013.
      "For deep archival storage, the ideal is to be able to create the desired artifact, then store it and forget about it, knowing that whenever we wish to access it again, it will still be there, and we will still be able to read or observe it." Using ancient historical materials as examples, archival storage should be made from materials which either do not oxidize (such as gold), or else are by nature fully oxidized or chemically-reacted materials. 

Merely making a mark in a layer of recording material is not robust data storage solution. By using layers of inorganic, stable materials, a digital medium can be created for recording and playback. Optical data recording has a significant advantage when it comes to data longevity because accessing the object does not involve any contact between the media and the recording and playback mechanism. This means the file can be read an infinite number times without any degradation. "And the relative simplicity of the playback mechanism means that, if the data persists on the media, future optical playback systems will readily be capable of being adapted as necessary to read data stored permanently on optical discs".

A materials approach to the problem of permanent digital data storage shows that by using extremely durable materials they can create a permanent storage medium. Permanent, in this case, does not mean 'forever', since "nothing that science acknowledges is permanent". The paper uses the ANSI/AIIM definition of permanent: “Life Expectancy: Length of time that information is predicted to be retrievable in a system under extended-term storage conditions. Rating for the ‘life expectancy’ of recording materials and associated retrieval systems. NOTE: The number following the LE symbol is a prediction of the minimum life expectancy in years for which information can be retrieved without significant loss when properly stored under extended-term storage conditions, e.g., LE-100 indicates that information can be retrieved for at least 100 years of storage."

The authors propose that a practical definition for permanent digital data storage would be: "Permanence: The ability of a digital data storage medium to last at least two hundred years without significant deterioration under normal use and storage conditions in libraries and archives. This means there is a 99.99% confidence of complete data recovery using the intended read mechanism or hardware."

Currently the only digital medium that can last for more than a decade or so is the M-Disc, from Millenniata, Inc. All other digital data storage media have very serious limitations when it comes to permanence. How can the life expectancy of a medium be established? One way is using the long accepted methods of accelerated testing using the Arrhenius and Eyring equations as the scientific foundation.

The paper looks at solid-state storage media and computer tape and explains why they are not permanent. None of the current solid-state storage options are viable for permanent digital data storage. The problem with magnetic media is that "the magnetic domains begin to relax, slowly reverting to their original random orientation, and slowly degrading the difference between the encoded 1s and 0s. Eventually, so many of these bits will have degraded that reading a file back will have become impossible." Magnetic tape uses an organic material as a binder, which will degrade with time; the recording layer separates from the polyester substrate causing permanent data loss. There is an urgent need for some way to store digital data permanently. The M-Disc is one viable solution and "we are well on our way to providing two additional media for permanent digital data storage."

Just having a permanent storage medium is not a complete solution without a way to read the data far into the future. But a permanent medium is indispensable for deep archival data storage. "Some future generation may struggle to learn how our data was stored, but if all the marks on the Rosetta stone had faded away, deciphering them would never have been possible. It is relatively easy and inexpensive to read the marks made on optical discs - the hardware is widely available and optical discs are the most widely adopted digital storage medium in history. And wide adoption is a powerful predictor of relative permanence of readability."

Related posts:

• Permanent Digital Data Storage: Tape, solid-state, and discs
• So what's up with the M-Disc? Part 1   Part 2
• Fighting entropy and ISIL, one image at a time
• Preserving Our Digital Heritage
• Dataliths vs. the digital dark age

Digital Preservation Matters.

So what's up with the M-Disc? Part 2

So what's up with the M-Disc? Part  2. Chris Erickson. 19 August 2015.
     Our library had some important digital images that could not be recreated if they were lost. I was a bit nervous about those, realizing that it was my responsibility to maintain them long term. We tested lots of potential solutions, including creating an in-house preservation system in about 2004.These things were all fine, but they didn't feel like a solution. With the M-Disc there was at least a possible solution and I wanted to find out if it was really better than what we were doing at the time.

The M-Disc was invented at BYU, and Barry Lunt had worked with the library on this. So the library was involved in the testing process from very early on. We went through a number of tests until we felt they were ready to use to preserve a major collection, and then setup a testing process. We migrated the Herculaneum papyrus images from gold CD to the M-Discs.

The burning and verification processes took more than an hour for each DVD. All images had been  examined beforehand to make sure they were good, then checksums for all the images were generated. We used the Nero software for burning and it generated helpful reports for each disc. Trying to burn discs from content across the network sometimes caused buffer errors, so we always put the content on the workstation and created an ISO image; the ISO image would then be burned to the disc.  After the disc was created, we verified the disc with Nero, with a disc check utility, and a custom routine that sampled random spots on the disc. We also had a disc analyzer that would do a low level check of the disc and report any read errors by the various categories. If there were read errors above an established threshold, the disc would be discarded and the images re-burned. The discs were tested on three separate computers. The final test was to copy the images from the discs to the workstation and then compare the checksums from the newly copied images to the checksums of the original images.  The tests were run right after burning, after one month, and then after one year. All tests and results were logged.  It was interesting to also see the tests that military did on optical discs, including the M-Discs, in the China Lake report.

From all the tests, we concluded that the M-Disc was the best long term storage option, and it is now used by several areas on campus. The library Digital Lab burns M-Disc copies of the images they create. They have 12 drives that can burn from either the PCs or Mac computers in the lab. The M-Discs are put in labeled sleeves in archival boxes, then accessioned into Special Collections. We also use the Rosetta software for our Digital Archive (tape backups and an annual tape archive are stored off site in the Granite Mountain Vault). The University Records department also uses the M-Discs, particularly for departments that want 'Permanent In Office' records.

The questions I often get asked:

1. Do I really think the M-Discs will last 1000 years? I can't image what computing will be like in 100 years, much less 1000 years. But I fully expect the M-Discs, with care, will last that long. Matthew Linford, a co-creator of the M-Disc, is a professor of Chemistry and Material Science. His published research papers show the M-Discs are made of inert materials that are not affected by environmental factors. The scientific materials approach shows there aren't any 'failure mechanisms' that would cause the discs to fail. It is unchanged by magnetism, temperature, light, humidity; with M-Discs, there is no possibility of bit rot or bit flips. The disc writing process actually makes irreversible physical pits on the M-Disc which will not fade over time. 
2. Will there be a way to read the M-Discs in 1000 years? I expect that DVD and Blu-ray discs will continue to be used for at least the next 25 years, if not longer. If you have the bits, there will always be a way to read them. The optical discs are written and read by standard documented technology, and the reading mechanisms, as one paper puts it, would be a trivial matter to build. 
3. Do the M-Discs require a special drive? Many optical drives (made by different manufacturers such as LG, Pioneer, Samsung, Panasonic, ASUS) can write the M-Discs. Since they follow the DVD and Blu-ray specifications, M-Discs can be read by any DVD or Blu-ray drive. The discs are available from mdisc.com, Amazon.com, Best Buy, or other places.
4. Isn't it difficult to burn a lot of optical discs? It is time consuming to burn lots of discs. The 100 GB M-Disc is available, and I am anxious to get the 200 GB M-Disc, so there would be only 5 discs per TB. I had suggested they build a 'tape library-like' device for the M-Disc so the discs would be easier to write. Hitachi LG Data Services has now built an Optical Archive System.  It can hold 1 PB of optical discs in a server rack. We have been testing the OAS in our library since about March, (I think it is great) but that is another post for another day.

Last point: The main reasons we use the M-Disc isn't because we need the data to last 1000 years. The main reasons are:

1. I want to decide when or if I will migrate the data to another medium. I don't want that decision to be made for me every few years based on an expected failure date for magnetic disk or tape. 
2. The longer the media lasts and the more reliable it is, the fewer times I have to migrate or check the data, and that saves money, time, and reduces the risk of data corruption.
   

And no, I don't own stock in the M-Disc. But I do use them at home for my family photos and documents.

Digital Preservation Matters.

So what's up with the M-Disc? Part 1

So what's up with the M-Disc? Part 1. Chris Erickson. August 18, 2015.
     There have been a number of questions about the M-Disc lately and I thought I would add my thoughts (without my work records here to jog my memory). I've been working with M-Discs since about the beginning. Some background: In the late 1990's a department on campus dedicated to the preservation of ancient texts began using multi-spectral imaging on unreadable ancient documents, such as Petra scrolls, Herculaneum papyrus scrolls, and others. The master images they produced were stored on gold CDs since these were supposed to last 300 years. However, our annual checks on the tens of thousands of CDs showed a yearly loss rate of 2-5% (we had several copies of all the discs so we could refresh them as needed). Looking for a way to resolve the failing discs led to discussions with Dr Barry Lunt, who created the M-Disc with Matthew Linford and others. We have tested the discs since 2008-2009 and gave feedback to the company (Millenniata). The first large collection we created was the Herculaneum images, which we migrated from CDs to the M-Disc DVD in February 2010. I continue to check the discs periodically and there haven't been any problems with them. The discs have worked well for us.

Here are few of my early posts about the M-Disc:

Millennial disc guarantees data preservation. Logan Bradford. Daily Universe. September 15, 2009.

Barry Lunt, a BYU information technologies professor, will launch a product with the company, Millenniata, that produces a disc just like a CD or DVD that will last up to 1,000 years. He learned, through his seven years working for IBM in computer data, that data on CDs and DVDs would decay and be lost over just a few years because of optical discs’ ephemeral qualities, such as when they are exposed to sunlight and humidity. [We have been testing these discs and writers.]

Startup crafts DVD-Rs for the 31st century. Rik Myslewski. The Register. 23 July 2009.

The Millenniata company has developed a new DVD-R technology that it claims will be readable for 1,000 years. The Millennial Disc Series is designed to eliminate the need for governments, financial institutions, libraries, and others to regularly refresh and rotate their digital-data collections. The data is etched into a "carbon layer with the hardness of a diamond". It requires a specialized writer and discs [but readable on any DVD player]. The discs are stable from minus 100° to plus 200° centigrade, and are dunked in liquid nitrogen as part of the testing. These discs are one element of a data preservation strategy.

Millenniata continues to make progress with its patent-pending Millennial Disc and Millennial Writer. Press Release. February 2, 2009. This press release has information about a new optical disc that has been developed. It is designed to be a permanent archiving product that has no degradable components and “safely stores data for 1,000 years”. The technology makes a permanent change to the disc. It is referred to as Write Once Read Forever™ and can be read in a standard DVD drive. [check back for test results.]

Digital Preservation Matters.

Permanent Digital Data Storage: Tape, solid-state, and discs

Permanent Digital Data Storage: An Overview. Barry M. Lunt, Matthew R. Linford, and Robert C. Davis. Brigham Young University. ISOM Conference. Received PDF August 2015. [From author's  version; not yet available on line.]
     Research shows that digital storage, whether optical, solid state, or tape, can be permanent and could potentially last over 100,000 years if permanent materials are used. The failure mechanisms are well documented. Knowing what materials to use to eliminate the failure mechanisms is the key to permanent digital storage.

Computer data storage has always been ephemeral because of the emphasis on density and speed. There has been little interest in developing a permanent way to store digital data. The authors, an engineer, chemist, and physicist, believe "that the optimal storage media does not need to be refreshed nor stored in special conditions, and that a store-and-forget approach (like printing books and storing them on shelves) is best because it is the simplest."

Permanent Storage Options and approaches

• Optical disks. The dominant failure mechanism is dye fading which can be removed by using permanent materials. 
• The media they developed (M-Discs) make permanent physical and optical marks on a standard DVD or Blu-ray format disc.
• Optical discs are a viable option for archival storage of large amounts of storage
• This permanent format is essentially guaranteed for many decades or centuries to come
• Hard disk drives are not permanent. The failure mechanisms, which are fairly well known, are predominantly mechanical. A materials approach cannot solve these problems.
• Solid-state storage. A materials approach has produced storage elements capable of lasting as long as integrated circuits; the failure rate of such circuits is measured in Failure In Time, or about 114,155 years. This is a permanent form of preservation. Their research has solved the dominant failure mechanism of early permanent programmable solid state storage.
• Permanent optical tape. Their materials-research shows that if correct materials are used, computer tape can also be permanent and the permanent tape would "match the density of LTO-5, allowing about 2 TB per cartridge. The price of the media should be equivalent to that of magnetic tape."

Optical discs, solid-state storage, and computer tape can all be made to store data permanently and last hundreds or thousands of years.

Related posts:

• How Long Is Long-Term Data Storage? 
• Dataliths vs. the digital dark age
• Millenniata Announces Results of ISO/IEC 10995 Standard Tests
• Start-up to release 'stone-like' optical disc that lasts forever

Digital Preservation Matters.

Preserving Our Digital Heritage

Preserving Our Digital Heritage. Barry M. Lunt, et al. IEEE. 2008.
     An early paper discussing the research in creating the M-Disc. Current storage technologies available for digital data: magnetic, optical, and solid state, have relatively short lifetimes, ranging from 5 to 30 years. Organizations aware of this problem store multiple copies of the data in multiple places, a strategy very useful where focused attention can be given to the data. But the world of consumer storage is characterized by a store and ignore mentality. The research they have been pursuing is investigate optically appropriate and chemically inert materials, then alter the material using laser irradiation, which leaves a permanent mark with high optical contrast. The recording layer is next to an adhesion layer (it is not an adhesive layer) which is a material which inherently adheres extremely well to polycarbonate. "The layers in our optical storage disc are made from materials that have withstood the test of time and are known to be capable of enduring over 1,000 years."


Related posts:

• How Long Is Long-Term Data Storage? 
• Now available: 100 GB capacity M-Disc
• Dataliths vs. the digital dark age
• Millenniata Announces Results of ISO/IEC 10995 Standard Tests
• Start-up to release 'stone-like' optical disc that lasts forever

Digital Preservation Matters.

How Long Is Long-Term Data Storage?

How Long Is Long-Term Data Storage? Barry Lunt. IS &T Archiving Conference. 2011.

    This paper looks at "In the context of archiving of physical documents, long-term storage has long been accepted to mean centuries." Since digital documents are much more ephemeral, archivists need to be aware of the technologies for preserving digital data and what can be expected for the long term. Results of several studies show that, with one notable exception, digital data cannot be expected to endure using any existing technologies.

Besides mechanical wear, oxidation, corrosion, and breaking of chemical bonds the most common failure mechanisms for materials, including digital data. Elevated temperature, humidity, and exposure to light increase the likelihood of failure, which is why archival storage relies on a controlled environment to reduce temperature, humidity and light.  The digital storage technologies, magnetic, solid-state, and optical, has known failure mechanisms, which have been studied. The studies of life expectancy for hard disk drives “eliminates HDDs as an archival storage option”.

The best way to guarantee that data will be readable in the future is by how widespread the adoption is. With data storage, "by far the most widespread formats in the history of digital data are the three main optical disc formats: CDs, DVDs and BDs". There are billions of optical readers in use today, and hundreds of billions of discs; no other digital storage technology even comes close.   While this does not guarantee long term persistence far into the future, the probability is extremely high. "If there is data still extant in these formats 500 years from now, it would be a relatively trivial matter to access that data".

Digital Preservation Matters

How do you preserve your personal data forever? We ask an expert

How do you preserve your personal data forever? We ask an expert. Simon Hill. Digital Trends. July 12, 2015.
     About two billion photos are uploaded to the cloud every single day. Every minute about 300 hours of video is uploaded to YouTube.  Our digital worlds are constantly expanding. Storage and access are not the same in the digital world as they are with physical materials.  The article is a discussion with Dr. Micah Altman, Director of Research and Head/Scientist, Program on Information Science for the MIT Libraries.  “Preservation is really about long term access. It’s about communicating with the future at some point.” Digital materials are under threat of loss: “One threat is that the media fails. The hard drive fails, the DVD fails, or the disc can’t be read. Another threat is that you can see the bits, but you can no longer tell what they mean because there’s no software available that will render that document... that format is not supported anymore.”

People think that digital files will last forever, but different types of digital media have "tremendously varying shelf lives". The shelf life can vary drastically by how they are stored. Not all storage is created equal. The media that’s sold to consumers is not really built for long term storage. “One of the challenges is that the media that’s usually sold to consumers, like hard drives in computers, are not really built for long term storage.... They’re designed for an operating lifetime of maybe three, four, or five years.” There is also a lot of variation in hard drive quality; some brands and batches last longer than others.

It’s possible to buy archival quality optical discs that are designed to last a long time. Some have been working on an Archival Disc standard. The  M-Discs from Millenniata have been designed to last for about 1,000 years. Many are looking at cloud storage, but you are "shifting the burden of figuring out how to preserve your data onto your cloud provider". Some vendors are Backblaze and CrashPlan, because they implement best practices and they’re transparent about what they do. Unfortunately, cloud storage isn’t a foolproof method of preservation, and you are placing a great deal of trust in the provider you select. Businesses go bankrupt or there could be other problems. It may help to use two cloud storage solutions, because the chance of simultaneous problems at two independent companies is much lower. There are also privacy questions to consider. The data can be protected with encryption, but "if you lose the encryption key then you’ve lost the data.”

Some other tips:

• Put materials in a standardized format. TIF and PDF-A are recommended
• Have an explicit succession plan plan in place for what happens to your accounts when you’re not in a position to access them
• Make multiple copies
• Research the cloud storage services before using them

There’s no definitive answer on how to ensure your digital files last forever, but you can hedge your bets and come up with a multi-pronged strategy to give your data the greatest chance of survival.


Related posts:

• Perspectives on Personal Digital Archiving - Digital Preservation.
• Helping Members of the Community Manage Their Digital Lives: Developing a Personal Digital Archiving Workshop
• ForgetIT 
• Preserving Your Personal Digital Memories
• Digital Preservation Matters - 21 March 2008: Rethinking Personal Digital Archiving

Digital Preservation Matters.

Storage Trends Around Computex 2015

Storage Trends Around Computex 2015. Tom Coughlin. Forbes. June 8, 2015.     The 2015 Computex Conference attracted many digital storage vendors. There were announcements about flash-based storage products, new memory products and optical archives.  CMC Magnetics said that it is selling 100 GB Blu-ray optical discs to Facebook for archiving applications. The company expects other internet service companies to follow suit. In May, Sony announced that it was buying Optical Archive, a start-up created by former Facebook executive. Sony is making a big push to create digital archiving solutions using Blue-ray disc technology and the acquisition is seen as an extension of this effort.

Related posts:

• Facebook uses 10,000 Blu-ray discs to store 'cold' data
• ARMA 2014: The Convergence of Records Management and Digital Preservation
• Fighting entropy and ISIL, one image at a time
• Dataliths vs. the digital dark age
• Safeguard the Future of Your Data: Digital Preservation Technology for the U.S. Federal Market.

 

Digital Preservation Matters.

Now available: 100 GB capacity M-Disc

Now available: 100 GB capacity M-Disc. Press release. Millenniata. June 2015.
     The new 100 GB Blu-ray M-discs are now available. The new disc has all the features of the  original M-Disc. Previously the products were the standard DVD and a 25 GB Blu-ray. Archiving large data sets is now much more convenient.
[The new 100 GB discs, which completely sold out in a short time, are now available again.]

Related posts:

• Millenniata Announces Results of ISO/IEC 10995 Standard Tests – Storage Newsletter
• Digital Preservation: Question and Response
• Fighting entropy and ISIL, one image at a time
• ARMA 2014: The Convergence of Records Management and Digital Preservation

Digital Preservation Matters.

Dataliths vs. the digital dark age

Dataliths vs. the digital dark age. Gary McGath. File Formats Blog. May 4, 2015.
Digital technology has allowed us to store more information at less cost than ever before, but in return this information is very fragile in the long term. The chances that your computer’s disk will be readable in a hundred years are poor. Information needs to be stored in a form that can survive long periods of neglect. We need dataliths; this strategy requires "a storage medium which is highly durable and relatively simple to read. It doesn’t have to push the highest edges of storage density. It should be the modern equivalent of the stone tablet, a datalith."

There are devices which tend in this direction. Millenniata, quartz glass data storage, and others. "Hopefully datalith writers will be available before too long, and after a few years they won’t be outrageously expensive. The records they create will be an important part of the long-term preservation of knowledge."


[M-Discs really are the only solution along these lines at present. They are long lived, inexpensive, easy to create and read, and created according to standards. -cle]

Digital Preservation Matters.

Fighting entropy and ISIL, one image at a time

Fighting entropy and ISIL, one image at a time. Whitney Blair Wyckoff. FedScoop. December 10, 2014.

United States security is generating so much data that traditional disk media is being pushed to its limits, requiring new technologies to safely store all that information. Hitachi Data Systems has a new technology to preserve information on disks in an infinitely expandable array. This platform uses Blu-ray XL M-DISCs that resist environmental conditions and can last for more than 1,000 years. The M-DISC optical solutions have proven survivability and durability. This system represents both "the highest reliability as well as the lowest overall cost of ownership representing superior savings in power, footprint and data reliability."

The IT can supplement magnetic storage with optical media to create a preservation tier that enables IT managers to migrate data when they want, not when the technology or media forces them.  This saves money and allows for more strategic long term planning. Flash media, magnetic tape storage, regular optical discs all are subject to deterioration and have short life spans. With additional storage servers, the amount of data that can be accessed in unlimited.

The system can preserve data for as long as necessary and access it whenever needed. Benefits provide lower operating costs through lower media migration costs, wider environmental storage requirements, migration-free technology upgrades and high media longevity and durability.

"The cost savings is stark while the possibility of data loss is virtually eliminated."

Digital Preservation Matters.

ARMA 2014: The Convergence of Records Management and Digital Preservation

ARMA 2014: The Convergence of Records Management and Digital Preservation. Howard Loos, Chris Erickson. October 2014. [PDF]
Presentation on records management and digital preservation given at the ARMA 2014 conference.
Notes:

• Records Management mission: To assist departments in fulfilling their responsibility to identify and manage records and information in accordance with legal, regulatory, and operational requirements
• RIM Life Cycle to DP Life Cycle
• Challenges and successful approaches
• Storing records permanently with M-Discs
• Introduction to Digital Preservation, challenges, format sustainability, media obsolescence, metadata, organizational challenges,
• Life of digital media
• Best practices and processes
• OAIS model
• Rosetta Digital Preservation System
• Library of Congress Digital Preservation Outreach & Education (DPOE) Network

Digital Preservation Matters.

Safeguard the Future of Your Data: Digital Preservation Technology for the U.S. Federal Market.

Safeguard the Future of Your Data: Digital Preservation Technology for the U.S. Federal Market. Hitachi brochure. 2014.
Hitachi’s Digital Preservation Platform (HDPP) is a non-magnetic storage solution that has the ability to preserve unlimited amounts of data for decades on end with minimal migration. The projected capacity of the storage solution is 1 PB per rack by the end of 2014. Offline media is also supported.

Cost-efficiency is another factor when considering long-term preservation. Traditional archives use a migration strategy that requires regular media refreshing which has proven to be costly over time. Migration is an ongoing process that takes a significant amount of resources.

Blu-ray optical media and M-DISC media ensure longevity and compatibility across generations of technology so the data can still be accessible as formats continue to evolve. Blu-ray discs are projected to 1 TB per disc. Mdisc capability is currently at 25 GB per disc, with plans for 300 GB per disc. Brochure also includes quick specs and diagrams.
 

Digital Preservation Matters.

Millenniata Announces Results of ISO/IEC 10995 Standard Tests – Storage Newsletter

Millenniata Announces Results of ISO/IEC 10995 Standard Tests – Storage Newsletter. Press Release. StorageNewsletter.com. 2013.06.10.
Millenniata, Inc. announced the results of its longevity test program based on the ISO/IEC 10995 standard. The results showed the median expected life of the M-DISC DVD was 1,332 years.  The same tests showed that other archival DVDs have an expected life of only 2.7 to 3.0 years.

Digital Preservation Matters.

Digital Preservation: Question and Response

Question: We are trying to move away from multiple physical copies of our archival audio.  Our goal is to just have redundant digital copies and one physical copy on some medium in case of catastrophic failure with the servers. What physical medium other audio archives are using for preservation/backup copies of digital audio files.

Response: It is important to have redundant digital copies in case of failure; we have had a number of occasions where we needed to recover objects from a second or third copy when other copies failed.  There are different ways to accomplish multiple digital copies, such as copies on server, tape (LTO), cloud (DuraCloud and others), and optical discs, or a mixture. There are advantages and disadvantages with each option; we have a mixture of these, but are moving towards two options, server copies with tape backup, and M-DISCs.  We have started using the Rosetta software to manage the server copies, which will have tape copies stored in a secure off site location.  But I want an additional copy on another medium, which is our M-DISC archive.

In general optical CDs and DVDs have a short lifespan.  For a number of years, I have been caring for a large disc archive of gold CDs and DVDs; there are multiple copies for redundancy. (There are tape copies also, which have had problems as well, but the gold archival discs have been considered the preservation copies.) Multiple copies are necessary since a percentage of the discs fail each year (the percentage depends on the collection).  In order to solve that problem, two professors on campus (Information Technology, and Chemistry & Material Science) created a digital storage medium that does not fail over time and is unaffected by any normal factors, such as light, heat, cold, oxidation, pollutants, material decay, bit flips, etc.  (Extreme stress tests by the US military could not get the discs to fail.)

The university licensed the technology to a company called Millenniata which has partnered with LG and others to produce the M-DISC.  Currently the M-DISC is a DVD format, which is somewhat of a drawback if you want to store large archives, but they are developing other densities and options. (There are some organizations using a multi-terabyte Millenniata device, but I have not seen it.) I have tested and used the M-DISCs for several years and have not had any problems.  (I plan to start another round of testing on my M-DISC archive this year to check on the status of the discs; I check them for usability, read error levels, and bit transfer integrity.) So I consider this my ‘copy in case of catastrophic failure’, while the Rosetta archive is my ‘active preservation archive’. There are others in the library and on campus (digital lab, records management, etc) that use the M-DISCs for their own long term copy for images, documents, audio, video, and such. Whatever you choose, you should think about multiple copies in multiple places on multiple media.  What you choose needs to fit your circumstances and be sustainable for your program. 

 Digital Preservation Matters.

Start-up to release 'stone-like' optical disc that lasts forever

Start-up to release 'stone-like' optical disc that lasts forever.  Lucas Mearian. Computerworld. August 8, 2011.

Millenniata has partnered with Hitachi-LG Data Storage to launch an M-Disc read-write player in early October.  Any DVD player maker will be able to produce M-Disc machines by simply upgrading their product's firmware. Millenniata said it has also proven it can produce Blu-ray format discs with its technology - a product it plans to release in future iterations. Currently the discs write at  only 4x speed, but they are working to increase it.

Millenniata partnered with Hitachi-LG Data Storage to provide M-Ready technology in most of its DVD and Blu-ray drives. The technology is addressing the needs of the long-term data archive market.  This disc does not need special temperature or humidity controls.