Our library had some important digital images that could not be recreated if they were lost. I was a bit nervous about those, realizing that it was my responsibility to maintain them long term. We tested lots of potential solutions, including creating an in-house preservation system in about 2004.These things were all fine, but they didn't feel like a solution. With the M-Disc there was at least a possible solution and I wanted to find out if it was really better than what we were doing at the time.
The M-Disc was invented at BYU, and Barry Lunt had worked with the library on this. So the library was involved in the testing process from very early on. We went through a number of tests until we felt they were ready to use to preserve a major collection, and then setup a testing process. We migrated the Herculaneum papyrus images from gold CD to the M-Discs.
The burning and verification processes took more than an hour for each DVD. All images had been examined beforehand to make sure they were good, then checksums for all the images were generated. We used the Nero software for burning and it generated helpful reports for each disc. Trying to burn discs from content across the network sometimes caused buffer errors, so we always put the content on the workstation and created an ISO image; the ISO image would then be burned to the disc. After the disc was created, we verified the disc with Nero, with a disc check utility, and a custom routine that sampled random spots on the disc. We also had a disc analyzer that would do a low level check of the disc and report any read errors by the various categories. If there were read errors above an established threshold, the disc would be discarded and the images re-burned. The discs were tested on three separate computers. The final test was to copy the images from the discs to the workstation and then compare the checksums from the newly copied images to the checksums of the original images. The tests were run right after burning, after one month, and then after one year. All tests and results were logged. It was interesting to also see the tests that military did on optical discs, including the M-Discs, in the China Lake report.
From all the tests, we concluded that the M-Disc was the best long term storage option, and it is now used by several areas on campus. The library Digital Lab burns M-Disc copies of the images they create. They have 12 drives that can burn from either the PCs or Mac computers in the lab. The M-Discs are put in labeled sleeves in archival boxes, then accessioned into Special Collections. We also use the Rosetta software for our Digital Archive (tape backups and an annual tape archive are stored off site in the Granite Mountain Vault). The University Records department also uses the M-Discs, particularly for departments that want 'Permanent In Office' records.
The questions I often get asked:
- Do I really think the M-Discs will last 1000 years? I can't image what computing will be like in 100 years, much less 1000 years. But I fully expect the M-Discs, with care, will last that long. Matthew Linford, a co-creator of the M-Disc, is a professor of Chemistry and Material Science. His published research papers show the M-Discs are made of inert materials that are not affected by environmental factors. The scientific materials approach shows there aren't any 'failure mechanisms' that would cause the discs to fail. It is unchanged by magnetism, temperature, light, humidity; with M-Discs, there is no possibility of bit rot or bit flips. The disc writing process actually makes irreversible physical pits on the M-Disc which will not fade over time.
- Will there be a way to read the M-Discs in 1000 years? I expect that DVD and Blu-ray discs will continue to be used for at least the next 25 years, if not longer. If you have the bits, there will always be a way to read them. The optical discs are written and read by standard documented technology, and the reading mechanisms, as one paper puts it, would be a trivial matter to build.
- Do the M-Discs require a special drive? Many optical drives (made by different manufacturers such as LG, Pioneer, Samsung, Panasonic, ASUS) can write the M-Discs. Since they follow the DVD and Blu-ray specifications, M-Discs can be read by any DVD or Blu-ray drive. The discs are available from mdisc.com, Amazon.com, Best Buy, or other places.
- Isn't it difficult to burn a lot of optical discs? It is time consuming to burn lots of discs. The 100 GB M-Disc is available, and I am anxious to get the 200 GB M-Disc, so there would be only 5 discs per TB. I had suggested they build a 'tape library-like' device for the M-Disc so the discs would be easier to write. Hitachi LG Data Services has now built an Optical Archive System. It can hold 1 PB of optical discs in a server rack. We have been testing the OAS in our library since about March, (I think it is great) but that is another post for another day.
- I want to decide when or if I will migrate the data to another medium. I don't want that decision to be made for me every few years based on an expected failure date for magnetic disk or tape.
- The longer the media lasts and the more reliable it is, the fewer times I have to migrate or check the data, and that saves money, time, and reduces the risk of data corruption.
No comments:
Post a Comment