Thursday, September 10, 2015

Testing a Permanent Digital Storage Archive – Part 1

Testing a Permanent Digital Storage Archive – Part 1. Chris Erickson. September 9, 2015.
     For about the past seven years I have been testing the M-Disc permanent digital storage media. The M-Disc is designed to use inert, permanent materials, so that digital data written to the discs are permanent.

The original M-Discs were in the standard DVD format that can hold up to 4.7 GB. This works well for MB or GB collections but it becomes a bit cumbersome when working with many TBs. It takes 213 DVDs to hold 1 TB of data. (I am looking forward to the 200 GB M-Discs, which would be 5 discs per TB.)

I suggested to the people at Millenniata that it would be much more useful for large data archives if they could take a tape library and replace the tapes with M-Discs. As long as I was wishing, I would like it to include some additional features:
  • network accessible
  • drag and drop capability to speed up the file ingesting
  • disc spanning to make migrating old CDs easier, as well as storing large files
  • fixity checking as a built in process
  • an index of the system for retrieval and management
  • automatic replication of discs to make multiple copies
  • ability to migrate to new discs with a higher capacity when they became available.  
And, of course, increase the disc capacity and decrease the price. Then we would have the perfect permanent storage.

Well, the capacity has increased from 4.7 GB to 100 GB per M-Disc and that will hopefully double soon. The price has dropped 10 fold. And Hitachi LG Data Storage (HLDS) has actually built the storage system. We’ve been talking with them since 2012, and so in February of this year they installed a test system at our library, the HLDS Optical Archive System (OAS). The storage system, which fits in a server rack, can hold up to 1 PB of archival storage space; we were testing a single storage library with 100 TB of storage space. The OAS uses a server for the processing, indexing and ingest functions. There is a YouTube video available.

The OAS uses long term Blu-ray discs; each is 200 GB. The optical discs we were testing were their standard discs which have been tested and certified for 50+ years. Millenniata hopes to have their 200GB discs available soon, and those could be used as well.  These are industry standard format Blu-ray optical discs. We were interested in these for long term storage because of their longevity and reduced operating cost, since they don’t have to be continually spinning. (Blu-ray Disc is increasingly used as an archival storage solution because of the longevity of the discs and the cost savings). The testing was supported by the HLDS group in California and they were extremely helpful with installation, training, frequent visits to our library, and weekly phone calls.

The Optical Archive System met all the functions on my wishlist. The question now was: How will it work with our Rosetta digital preservation system?
(Updated to clarify and answer questions.)

Related posts:


Hamel Reyes said...

Very good information dear, you have described all the procedure properly. High density server is required to storage data safely. The storage servers are usually the Network Attached Servers. There are many advantages of the storage server and storage server accessories.

rackmountpro said...

Thanks for sharing this useful information with us. Digital server can use virtual private server platforms, but this create several problems. With cloud server, users manage systems, not individual servers. Get storage servers at Rackmountpro. said...

Great article. I found it very informative and impressive. Thank you for sharing . I was in a search of something like this. It was quite helpful for me. Nice work done