Measure for Measure: Tracking Effort in Born Digital Processing

Measure for Measure: Tracking Effort in Born Digital  Processing. Erika Farr. CurateGear 2015. January 7, 2015.
     Presentation at CurateGear on measuring and tracking efforts. The percent of effort for each task was helpful. Some notes from the slides:

FY2014 Metrics Gathering

• Three Digital Archives staff members processing born digital collections
• Established categories for measuring effort
• Obtain Data, Virus Check
• Ingest
• Appraisal and Developing Processing Plan
• Collection Processing (Arrangement and Description, Privacy and Restrictions)
• Prepare for Access 
• Used Redbooth for business collaboration tool

Processing Metrics in FY2014

• Processed born-digital material in 5 collections
• Processed over 2 GB worth of data, over 4100 files
• 388 hours of effort
• On average the processing completed over 5MB per hour, 18 files per hour.

Digital Archives Effort by Task

• 4% Obtain Data
• 1% Ingest
• 19% Appraisal and Processing Plan
• 71% Collection processing
• 5% Prepare for Access

Metrics are useful for:

• Creating and modifying workflow and deciding the tools to use
• Business planning for acquiring collections, staffing, budgeting, and prioritizing 
• Determining the total cost of collections

Digital Preservation Matters.

Heroic Measures: Reflections on the Possibility and Purpose of Digital Preservation

Heroic Measures: Reflections on the Possibility and Purpose of Digital Preservation. David M. Levy. Xerox Palo Alto Research Center. Third ACM Conference on Digital Libraries.  June 23-26, 1998. [PDF requires a subscription to the journal.]
     A great article from the time when digital preservation was first being discussed and explored. In looking how to move our digital preservation program to the next level, this is a reminder that digital preservation is not about the technical variables but about object integrity and preserving the features that distinguish a work. It is about the use of the object and about the needs of the user, and about society learning how to find its way through life with digital information.

Quotes from the article:

• “Whatever preservation method is applied, however, the central goal must be to preserve information integrity; that is, to define and preserve those features of an information object that distinguish it as a whole and singular work.” The problem comes if one’s focus is exclusively, or too strongly on, the digital object, and at the expense of other considerations, such as use.
• In both the library world and the world of archives, people at times have become so focused on the artifacts themselves that they have risked losing sight of their users or their users’ needs.
• The digital object is that which produces the tangible, perceptible things which people then use. The digital object is necessary but not sufficient. How it is preserved - which features, which behaviors are deemed crucial - is all important, and if these decisions aren’t guided by use considerations, it will be to our detriment.
• “Viewed developmentally, the problem of preserving digital information for the future is not only, or even primarily, a problem of fine-tuning a narrow set of technical variables. It is not a clearly defined problem like preserving the embrittled books that are self destructing from the acid in the paper on which they were printed. Rather, it is a grander problem of organizing ourselves over time and as a society to maneuver effectively in a digital landscape.”

Digital Preservation Matters.

Permanent Storage: Sapphire discs

Permanent Storage: Sapphire disks. Chris Erickson. August 17, 2015.
     For several years there have been experiments with using sapphire discs for permanent storage and now a Kickstarter campaign to fund the disk. This seems similar to the Norsam HDRosetta archival preservation technology, which has been around for decades.

• The 10-million-year sapphire hard disk. Sebastian Anthony. ExtremeTech. July 13, 2012. The ultimate long-term storage solution: two 20cm (8in) sapphire disks, molecularly fused together, with a thin layer of platinum in between, and inscribe up to 40,000 miniaturized pages of text or images on the platinum. The disk is expected to have a lifetime of 10 million years.
• A Sapphire Hard Disk Will Last 1 Million Years (But You Can't Afford It). Jamie Condliffe. Gizmodo. July 13, 2012. 
       A team of scientists made a hard disk from sapphire which it claims will last 1 million years when looking to preserve records of where nuclear waste repositories were buried, not just in the near-future, but for tens of thousands of years. Currently, there's no digital format that we know can last that long with certainty. Two problems: First, the prototype disk cost $30,000 dollars. Second, what language to use when writing on the disk. 
• Fahrenheit 2451 - Preserve Your Data in a Sapphire Disk. Alain Rey & Farid Benzakour. Kickstarter. July 14, 2015. The project raised €45,461 with 341 backers. The Fahrenheit 2451 project has the ability for people to order their own nanoforms or jewelry with content inscribed. 
• It appears that the nanoform (composed of two slices of synthetic sapphire which can withstand 2451 degrees Fahrenheit) currently etches analog letters and images on a disk. At the smallest size, every character has a size of 10 μm, and the disk can be read with a digital microscope or similar enlarging device. Disks must be created and inscribed in a clean room environment. The current facility is Grenoble, France. 
• The 8'' nanoform can store up to 10,000 letter pages at 150 dpi or 650x850 pictures. The FAQ indicates it is possible to digitize the content again. It requires a professional scanner with 25,600 dpi precision, which you can find in most of labs and printing press, and the "quality of your pictures and documents will be approximatively the same as the original."  There is a viewing platform online to view the images. 
• The regular price of the 4 inch nanoform, with up to 2,500 images or pages at 7,000 MP, is $1,299.

This is certainly an interesting advance in technology and it is worth watching the progress, but this disk is not about digital preservation or curation. This is about End of Time Preservation rather than Timely Preservation.

A better model for permanent digital preservation storage is the M-Disc technology: creating a permanent 1,000 year disc in any computer in a few minutes, that can be read by any other computer, at a cost as low as $0.17 per GB.

Digital Preservation Matters.

Failure rates among consumer-class hard drives

Read this before you buy another hard drive. Lucas Mearian. Computerworld. January 2014 and updated September, 2014.   
     An article looking at Backblaze, a data storage service provider, who detailed the failure rates among the more than 27,000 consumer-class hard drives it uses in its data center. The data center contains Seagate, Hitachi and Western Digital drives, plus some others that they mention. They have specific information which shows that in a three-year period, 3.1% of  the 12,956 Hitachi drives failed; 5.2% of 2,838 Western Digital drives failed; and a sizable 26.5% of the 12,765 Seagate drives failed. They had a small number of Toshiba and Samsung drives. They buy the least expensive drives available.

"Hitachi does really well. There is an initial die-off of Western Digital drives, and then they are nice and stable. The Seagate drives start strong, but die off at a consistently higher rate, with a burst of deaths near the 20-month mark." The study includes data on 15 drive models totaling more than 12,000 drives each from Seagate and Hitachi, and almost 3,000 drives from Western Digital. There were also several dozen drives from both Toshiba and Samsung, but not enough for solid statistical results.

In an updated post on the Backblaze blog they report they have 34,881 drives and store over 100 petabytes of data. They also report "The surprising (and bad) news is that Seagate 3.0TB drives are failing a lot more, with their failure rate jumping from 9% to 15%. The Western Digital 3TB drives have also failed more, with their rate going up from 4% to 7%."
As shown in the chart on the blog, the HGST (formerly Hitachi) drives have the lowest failure rate of all.

The question was asked if they were going to switch to enterprise drives. "The assumption that “enterprise” drives would work better than “consumer” drives has not been true in our tests. I analyzed both of these types of drives in our system and found that their failure rates in our environment were very similar — with the “consumer” drives actually being slightly more reliable."

Digital Preservation Matters.

Born-Digital Access in Archival Repositories: Mapping the Current Landscape

Born-Digital Access in Archival Repositories:Mapping the Current Landscape. Preliminary Report. Rachel Appel, et al. Google docs. August 2015.
     This is a document outlining research that will be presented at the SAA meeting and it contains research and a list of resources that looks at how archival best practices for processing and preserving born-digital materials have developed over the last decade. It says there are no "established best practices for providing research access to born-digital materials that scale to match the volume of born-digital material and meet archival standards surrounding authenticity of records, descriptive metadata, and the protection of donor privacy and intellectual property." Archivists in various institutions face pressure to accept born-digital records, but often they have neither the" preparation nor the resources to preserve them or provide access to them.” One of the problems is that  archivists are not involved early enough in the acquisition process. Another is that they are not always consulted when policy decisions are made at the institution.  The biggest challenge access is the "sensitivity of materials--concerns about copyright, confidentiality, privacy,  intellectual property, and personally identifiable information. The second biggest challenge is IT infrastructure and file size structure, or rather, the lack of it (28 respondents).”

The qualitative and quantitative data were examined to pinpoint aspects of born-digital access that participants classified as gaps: areas participants considered highly important in which little or no practical progress has been made. The gaps clustered into the following five overarching themes, listed in order of frequency.

1. Gaps in Tools and Systems
2. Gaps in Business Analysis, Resource Allocation, and Advocacy
3. Gaps in Skills for Archivists, Sharing Information, and Training Each Other
4. Gaps in Understanding Users
5. Gaps in Research and Policy

The five most common aspects of born-digital access that participants were planning to implement were:

1. Access in Reading Room, Remote, and Online (42 mentions)
2. Metadata for Access and Processing (25 mentions)
3. Creation of Copies and Images (19 mentions)
4. Privacy and Redaction (14 mentions)
5. Tools (8 mentions)

Digital Preservation Matters.

UK High Court: Format changes, backups, and private copies are illegal

Ripping music and films illegal again after High Court overturns new law. BBC Newsbeat. 17 Jul 2015.
     The UK government introduced a law, Copyright and Rights in Performances (Personal Copies for Private Use) Regulations 2014 in October 2014, meaning it was legal to transfer music from one format to another in your home library and to make back-ups for personal use. (It was still illegal to share those copies with friends or family or to sell on that music or data). But that law has been overturned in the UK High Court which affects at least CDs, DVDs, Blu-rays, MP3s, and e-books.
The High Court said the government acted unlawfully by introducing an exception to copyright for private copying without fair compensation. The change in the law has implications for teachers who use copyright materials on interactive whiteboards and writers who quote other sources.

The UK just made iTunes illegal. Buster Hein. Cult of Mac. August 6, 2015. 
     In a discussion with the UK Intellectual Property Office, they said “It is now unlawful to make private copies of copyright works you own, without permission from the copyright holder – this includes format shifting from one medium to another. It includes creating back-ups without permission from the copyright holder as this necessarily involves an act of copying.” 

[This will likely affect libraries, archives and museums who may receive donations of computers and media; they would need to ensure before processing that there are no copyrighted materials included.]

Digital Preservation Matters.

PDF/A Flyer

PDF/A.  PDF/A Competence Center. June 18, 2015. (PDF version of the Flyer).
     PDF/A is the ISO standard for archiving electronic documents using the PDF format. It required years of cooperation between software developers, industry associations and government agencies. There are three parts to the standard which were published between 2005 and 2012. Parts 2 and 3 add options for combining several PDF/A files into one PDF/A collection, embedding the PDF’s source files or other data, support for transparency, and more.

PDF/A-2 is defined by ISO 19005, which "provides a mechanism for displaying electronic documents in such a way that the visual image is maintained over time, irrespective of the tools and systems used for their production, storage and reproduction”. It does not define an archiving strategy; instead, PDF/A specifies technical requirements for PDF electronic documents to ensure reliability after the file’s creation.

Since 2005 PDF/A has become the preferred format for archiving electronic documents. The PDF format, standardized in 2008 as ISO 32000-1, is used by many people. PDF will continue to ensure reliable access to PDF documents, making PDF ideal for long-term archiving.

Digital Preservation Matters.

Advocacy and Born-Digital Archives

Advocacy and Born-Digital Archives.  Mike Shallcross.  Bentley Historical Library. August 13, 2015.
     What is it that you do, anyway? When talking to library/university administrators and outside financial donors, we need to adjust how we explain what we do with digital archives. They may not need to know all the little details, but they really need to know it is that we do and why it's important. If we can't do that then getting resources and being relevant will be an uphill battle. Advocacy is an integral archives as we seek to show the value and get support.  We have to represent our work in a way that others can really understand.

We have to make a case, and the way that we present it may be different for donors and for administrators. "Successful advocacy should involve and inspire the audience so that they understand how and why they might benefit from our work." Show the added value of the archives to the people and the organization, show the benefits in a quantitative way. The value may be:

• Helping researchers understand the collections and improve the way they use them
• Ensure that content [which are valuable assets] can still be accessed in the future
• Protecting personal information, intellectual rights, institutional policies, legal requirements, and donor agreements.
• Developing resources for researchers to use and reuse materials

We must show the benefits of this if we want others to commit resources to the archives. Highlight innovations, achievements, and recognitions.

"Our work in “digital curation” encompasses traditional archival functions—the process of selecting materials of high research or intrinsic value and making them accessible to researchers—but also involves additional steps to ensure the integrity and authenticity of content."

[It also helps to show the value in money: money earned, money saved, money that can be put elsewhere.]

Digital Preservation Matters.

Permanent Digital Data Storage: Tape, solid-state, and discs

Permanent Digital Data Storage: An Overview. Barry M. Lunt, Matthew R. Linford, and Robert C. Davis. Brigham Young University. ISOM Conference. Received PDF August 2015. [From author's  version; not yet available on line.]
     Research shows that digital storage, whether optical, solid state, or tape, can be permanent and could potentially last over 100,000 years if permanent materials are used. The failure mechanisms are well documented. Knowing what materials to use to eliminate the failure mechanisms is the key to permanent digital storage.

Computer data storage has always been ephemeral because of the emphasis on density and speed. There has been little interest in developing a permanent way to store digital data. The authors, an engineer, chemist, and physicist, believe "that the optimal storage media does not need to be refreshed nor stored in special conditions, and that a store-and-forget approach (like printing books and storing them on shelves) is best because it is the simplest."

Permanent Storage Options and approaches

• Optical disks. The dominant failure mechanism is dye fading which can be removed by using permanent materials. 
• The media they developed (M-Discs) make permanent physical and optical marks on a standard DVD or Blu-ray format disc.
• Optical discs are a viable option for archival storage of large amounts of storage
• This permanent format is essentially guaranteed for many decades or centuries to come
• Hard disk drives are not permanent. The failure mechanisms, which are fairly well known, are predominantly mechanical. A materials approach cannot solve these problems.
• Solid-state storage. A materials approach has produced storage elements capable of lasting as long as integrated circuits; the failure rate of such circuits is measured in Failure In Time, or about 114,155 years. This is a permanent form of preservation. Their research has solved the dominant failure mechanism of early permanent programmable solid state storage.
• Permanent optical tape. Their materials-research shows that if correct materials are used, computer tape can also be permanent and the permanent tape would "match the density of LTO-5, allowing about 2 TB per cartridge. The price of the media should be equivalent to that of magnetic tape."

Optical discs, solid-state storage, and computer tape can all be made to store data permanently and last hundreds or thousands of years.

Related posts:

• How Long Is Long-Term Data Storage? 
• Dataliths vs. the digital dark age
• Millenniata Announces Results of ISO/IEC 10995 Standard Tests
• Start-up to release 'stone-like' optical disc that lasts forever

Digital Preservation Matters.

Asimov, Archives and Records

Asimov, Archives and Records. Chris Erickson. August 12, 2015.
     When I started working with digital preservation, a co-worker referred me to Isaac Asimov's Foundation series. Several passages in two of the books sounded very much like what we are dealing with now, though the time frames are much longer.  Here are some quotes from those books I thought interesting:

Foundation, pp. 34-35.

• "Q. How do you propose to do this?"
• "A. By saving the knowledge of the race. The sum of human knowing is beyond any one man; any thousand men. With the destruction of our social fabric, science will be broken into a million pieces. Individuals will know much of exceedingly tiny facets of what there is to know. They will be helpless and useless by themselves. The bits of lore, meaningless, will not be passed on. They will be lost through the generations. But, if we now prepare a giant summary of all knowledge, it will never be lost. Coming generations will build on it, and will not have to rediscover it for themselves...." 
•  "Why, then, should we concern ourselves with events of three centuries distance?” 
• "I shall not be alive half a decade hence,” said Seldon, “and yet it is of overpowering concern to me. Call it idealism. Call it an identification of myself with that mystical generalization to which we refer by the term, ‘humanity.’ ”

Foundation and Earth, pp. 5, 11, 15.

• "Those myths and legends are all there are. There are no actual records, no documents.”
• “Documents twenty thousand years old? Things decay, perish, are destroyed through inefficiency or war.”
• “But there should be records of the records; copies, copies of the copies, and copies of the copies of the copies; useful material much younger than twenty millennia. They have been removed."
• "Those documents are referred to in known historical records, but the documents no longer exist in the Galactic Library. The references to them may exist, but any quotations from them do not exist.”
• "You cannot have a reasonable civilization without records of some kind."
• "A civilization in being is not likely to destroy its early records. Far from judging them to be archaic and unnecessary, they are likely to treat them with exaggerated reverence and would labor to preserve them."

(And another site with similar quotes). Preservationists aren't looking that far ahead yet, but with the research into permanent storage technology that is being done, records and archives could be permanent.

Digital Preservation Matters.

Keeping the bits

Keeping the bits. Chris Erickson. August 11, 2015.
     Some time ago I wrote a paper about my view of preservation titled the "Three Pillars of Digital Preservation". The first and most important of the three pillars was 'keeping the bits'. If you don't have the bits, you are out of luck. If you do have the bits, there is a good chance you can find ways to access them and understand them.

For many, including the preservation world, it seems archiving equates with storing content on the cloud for 5 or 10 years, or making lots of copies that when neglected become lots of confusion. We (consumers,  society, preservationists,  everyone.) store our permanent records on temporary media, or we put them in the cloud on someone else's temporary media. Preservation seems to be aimed at processes attempting to avoid or manage the effects of the problem instead of solving the problem itself.

We should be advocating for permanent solutions for storage,  for formats, for file management and for access.  There are a number of groups working on permanent storage solutions; some are available now, but unless we choose to use them or make them a priority, the technology may never be used, edged out by cheaper 'solutions'. Nothing will ever be permanent until we choose that as our standard.

Digital Preservation Matters.

FY2014 Preservation Statistics Report

 FY2014 Preservation Statistics Report. Annie Peterson, Holly Robertson, and Nick Szydlowski. American Library Association. August 10, 2015. [PDF report]
     The majority of the survey respondents were academic libraries (64% ). The results are for review and analysis. The Preservation Statistics Survey no longer distinguishes between reformatting and digitization for access versus preservation. It encourages preservation units and digital initiatives units to cooperate and foster ongoing, mutually beneficial relationships.

• The figures show that 80% of AV items digitized were magnetic media. AV preservation is urgent and more research is needed to understand the issues.
• Microfilm accounted for 60% of the non-print items digitized.
• Most participants (62%) responded that digital preservation responsibilities take place outside of the preservation unit.  The majority of those have digital initiatives units that are responsible for the digital preservation activities. Others indicated that digital preservation is an activity spread across multiple units, while the remaining institutions report that digital preservation activities are the responsibility of a collections management or technical services unit or in the smallest group it is part of an IT / Systems unit. 

Respondents are managing approximately 10 Petabytes of data and added over 26 million items to their digital repositories in FY2014. Web archiving accounted for 60% of the items added to digital repositories.

Digital Preservation Matters.

Keeping Data For A Long Time

Keeping Data For A Long Time. Tom Coughlin. Forbes. June 29, 2014.
     Keeping information for a long time has always been a challenge.  Thermodynamics doesn’t favor information lasting a long time and so to make that happen people have to spend effort and energy. Deciding how to create a long-term archive involves choosing the right storage system with the right technology under the proper environmental conditions.  This can be combined with migration and replication practices to improve the odds of keeping content useful and accessible for an extended period of time. A conference looked at digital storage for long term archiving and preservation.Some of the technologies:

• It appears conventional flash memory may not have good media archive life and should only be used for storing transitory data
• Hard disk drives are used in active archives have problems because they wear out and even if the power is turned off the data in the hard disk drive will eventually decay due to thermal erasure  
• Digital magnetic tape under low temperature/humidity is a good candidate for long-term data retention  
• Optical storage has also been used for long-term data retention and should last at least several decades. Facebook has a 1 PB prototype that should reduce the storage costs by 50% and the energy consumption by 80% of their Hard Disk storage
• Sony said their properly made archival grade optical discs should have a shelf life of 50 years.
• Hitachi Data Systems showed costs for 5 PB of content over 75 years is less than frequent tape and HDD replacement.

A lot of digital data has persistent value and so long term retention of that data is very important. It is estimated the storage for archiving and retention is currently a $3B market growing to over $7B by 2017. "Magnetic tape and optical disks provide low cost long-term inactive storage with additional latency for data access vs. HDDs due to the time to mount the media in a drive.  Thus depending upon the access requirements for an archive it may be most effective to combine two or even three technologies to get the right balance of performance and storage costs."

Related posts:

• How Long Is Long-Term Data Storage? 
• Storage Trends Around Computex 2015
• Preserving Our Digital Heritage

Digital Preservation Matters.

Library science: The word on our archival future

Library science: The word on our archival future. Michael Lesk. Nature. 23 April 2015.
     Most libraries are facing a “perfect storm”: decreasing use, inadequate resources, and users satisfied with easy access to online information. US research libraries saw a 69% decline in the number of queries handled between 1991 and 2012. Academic library budgets are shrinking: from 3.7% of the total budget of a typical US university in the early 1980s to 1.8% in 2011. About half of an academic library budget is spent on acquisitions, which is increasingly consumed by journals. However, open-access publishers such as the Public Library of Science (PLOS) provide information with no reader fee, changing the economics entirely. School libraries, which still have a key role in teaching scholarly habits, are suffering. “The school library should be as important as the school sports team.” The increasing commercialization of information raises barriers. Digital preservation of library holdings is riskier than traditional methods.

Will governments recognize the importance of libraries and librarians?  Political will is essential, as is innovative energy among librarians. Palfrey hopes that conventional and new library technologies will sit side by side. This is not common in technological change: car-rental companies do not run livery stables. I suspect that a more likely future is that libraries (and museums) will be divided into the 'wholesalers' that have large historic collections, such as the US Library of Congress or the British Library, and the 'retailers' such as university libraries, which serve faculties.

Related posts:

• Libraries are places of knowledge creation and librarians our educators
• Forecasting the Future of Libraries 2015 
• Robert Darnton closes the book
• Falling Though the Cracks: Digital Preservation and Institutional Failures
• Why Libraries Matter More Than Ever in the Age of Google
• Libraries could outlast the internet, head of British Library says
• Are libraries sustainable in a world of free, networked, digital information?

 

Digital Preservation Matters.

Digital Preservation Tools on Github.

Digital Preservation Tools on Github. Chris Erickson. Blog. August 2015.
     While looking for a particular tool I came across several others that look interesting. I have not yet tried them, but this is a reminder that I need to check into them. 

• epubcheck: a tool to validate EPUB files. It can detect many types of errors in EPUB. OCF container structure, OPF and OPS mark-up, and internal reference consistency are checked. EpubCheck can be run as a standalone command-line tool or used as a Java library.

• preservation-tools: Bundles a number of preservation tools for all file types and tools in a modular way. Includes:
• PdfHeaderChecker (able to detect the software used to create a PDF),
• PdfAValidator (Checks via PDFBox if a PDF/A is valid. Runs through a folder and picks out only PDF/A-files),
• iTextRepairPdf (take a PDF-file and copies the content page-per-page to a new, PDFA1-conform PDF-file)
• PdfToImageConverter (Converts PDF Files in a certain folder to JPEGs page-per-page)
• PdfTwinTest (compares the two PDF line-by-line and puts out differences. This is handy for after-Migration Quality-Checking)

• wail: Web Archiving Integration Layer (WAIL). A graphical user interface (GUI) atop multiple web archiving tools intended to be used as an easy way for anyone to preserve and replay web pages.

• db-preservation-toolkit. The Database Preservation Toolkit allows conversion between Database formats, including connection to live systems, for purposes of digitally preserving databases. The toolkit allows conversion of live or backed-up databases into preservation formats such as DBML or SIARD, XML-based formats created for the purpose of database preservation. The toolkit also allows conversion of the preservation formats back into live systems to allow the full functionality of databases. For example, it supports a specialized export into MySQL, optimized for PhpMyAdmin, so the database can be fully experimented using a web interface.

• DPFManager. DPF Manager is an open source modular TIFF conformance checker that is extremely easy to use, to integrate with existing and new projects, and to deploy in a multitude of different scenarios. It is designed to help archivists and digital content producers ensure that TIFF files are fit for long term preservation, and is able to automatically suggest improvements and correct preservation issues. The team developing it has decades of experience working with image formats and digital preservation, and has leveraged the support of 60+ memory institutions to draft a new ISO standard proposal (TIFF/A) specifically designed for long term preservation of still-images. An open source community will be created and grown through the project lifetime to ensure its continuous development and success. Additional commercial services will be offered to make DPF Manager self-sustainable and increase its adoption.

• PreservationSimulation. This project is to provide baseline data for librarians and researchers about long-term survival rates of document collections. We have developed computer simulations to estimate document failure rates over a wide variety of conditions. The data from these simulations should be useful to stewards of such collections in planning and budgeting for storage and bandwidth needs to protect their collections.

• flint.  Facilitate a configurable file/format validation. Its underlying architecture is based on the idea that file/format validation almost always has a specific use-case with concrete requirements that may differ from a validation against the official industry standard of a given format. The following are the principle ideas we've implemented in order to match such requirements.

• excel. Regarding the second issue: how to best retain formulas and other essential components of spreadsheets, like Excel, one of our data curators, John McGrory (U of Minnesota), just published a tool in GitHub that can help. In our data repository, we use the tool each time a dataset is submitted and zip these resulting files as the "Archival Version of the Data." Download the software at http://z.umn.edu/exceltool. See also a description of what the tool does: http://hdl.handle.net/11299/171966

Digital Preservation Matters.

Digital Heritage: Semantic Challenges of Long-term Preservation

Digital Heritage: Semantic Challenges of Long-term Preservation. Christoph Schlieder. Semantic Web – Interoperability, Usability, Applicability. 05/06/2010.
     This is an excellent article on long term preservation. It is argued that a period of 100 years constitutes an appropriate temporal frame of reference for addressing the problem of semantic aging. Ongoing format migration constitutes currently the best option for temporal scaling at the semantic level.

Digital preservation focuses on finding solutions that scale well along the temporal dimension. In the pre-digital world, the preservation of written records over long periods of time depended on several factors:

1. The record needs to be preserved physically
2. The ability to read the record and language need to exist
3. There must be a community that still shows interest in the record

There are a number of reasons why digital preservation may fail.

•  Media aging: Any medium that carries a digital encoding will physically deteriorate until it is no longer possible to recover the original bit stream.
• Semantic aging: The evolution of data formats and the fact that knowledge about data semantics quickly disappears if not specified explicitly.
• Cultural aging: The community loses interest in some content; the documents are no longer retrieved, and is not maintained and transmitted any more, its loss is almost unavoidable

It is important to identify the time scale with preserving digital items. In the short term, 10 years, there are any number of solutions. So that the short term problem "may be considered solved". The most  ambitious time frame for digital preservation is the one promoted by the Long Now Foundation, which is a formidable period of 10,000 years. This article looks at a more modest period, 100 years. The agenda for this period is:

1. find strategies to access digital contents from the past 50 years in spite of aging factors
2. plan the preservation of currently accessible digital content for future use the next 50 years

This problem cannot be solved just by "simply agreeing on a standard format for digital archiving". There are two digital preservation initiatives, migration and emulation. Migration seems to work best for a document-centered workflow, and emulation "constitutes the best solution for archives of highly interactive media". Digital preservation initiatives conceived of preservation as an ongoing process around a digital curation lifecyle model and incorporate migration strategies to reduce risks. This is the basic link between digital preservation and the semantic web. How well this works depends on how solutions scale over the 100 years. "Only by looking at periods that are significantly longer than the 10 years" can we tell how the solutions will work. The aging processes have always been at work. "The best way to overcome the effects of semantic ageing is by migrating digital records into new formats." Long-term preservation forces the research community to adopt a much longer temporal frame of reference."

"Taking cultural ageing seriously means to abandon the idea that digital preservation operates like a time capsule. The picture of content that is enclosed in a digital capsule to be opened at some moment in the future is misleading because it is not the past that sends messages to the future. Rather, it is the present that makes choices, selecting content from the past and linking to it. This ongoing process of linking from the present into the past makes up digital heritage." 

Related posts:

• The Twelve Principles of Digital Preservation (and a cartridge in a repository…)
• Investing in Curation. A Shared Path to Sustainability. Final RoadMap

• Preserving Our Digital Heritage
• How Long Is Long-Term Data Storage? 
• Storage Trends Around Computex 2015
• Fighting entropy and ISIL, one image at a time
• Dataliths vs. the digital dark age
• Millenniata Announces Results of ISO/IEC 10995 Standard Tests
• Start-up to release 'stone-like' optical disc that lasts forever

Digital Preservation Matters.

Another successful work meeting on ontologies

Another successful work meeting on ontologies. Johannes Biermann. Pericles Blog. 10 August 2015.
The Centre for Research & Technology, Hellas (CERTH) hosted a technical workshop on the Linked Resource Model (LRM), the LRM service, domain ontologies, ecosystem model and policies.
Discussion of the progress on the domain ontologies and how the concept of “resource” has been adapted. Other topics included:

• Art & Media domain ontologies
  Art & Media domain ontologies, to make them useable for the concrete scenario and for similar scenarios. 
• The Science domain ontology uses Topic Maps to organise the knowledge with index and thesaurus 
• The Ecosystem ontology has  a simplified graphical view of the ontology, with agents of different types, simplified dependency types,  significance of the resources, and refinement of the process and policy elements.
• Linked Resource Model (LRM), including changes and the new features
• Linked Resource Model service that includes dynamic parts that can be used to express time, activities, actions, versions and rules.
• Roles of policies and how they are handled, the tools that drive the policies and how they can be tested. 

Digital Preservation Matters.

CSV Validation Tool and API

CSV Validation Tool and API. Github. 2015.
     This tool (csv-validator) is a Validation Tool and APIs for validating CSV (Comma Separated Value) files by using CSV Schema or other schemas, including those used by the Digital Preservation department at The National Archives. It may be used as a stand-alone command line tool, a desktop tool, or as a library in a project. It will verify that the CSV Schema itself is syntactically correct. The tool supports MD5, SHA-1, SHA-2, SHA-3, and SHA-256 checksum algorithms.

Related posts:

• The ABCs of Digital Preservation
• Tabula
• Digital Preservation Tools 

Digital Preservation Matters.

Why PDF/A validation matters, even if you don’t have PDF/A

Why PDF/A validation matters, even if you don’t have PDF/A. Johan van der Knijff.  KB Research, National Library of the Netherlands. July 7, 2015.
     The PDF format has a number of features that don’t fit with the aims of long-term preservation and accessibility, such as encryption, password protection, external fonts and reliance on external software. Some examples are PDFs that use Quicktime content. Acrobat cannot render this format natively, and relies on an external player. Also files that use Linux fonts, or files with 3D content.
Institutions may want to check their PDF files to similar examples. Reasons for doing this include:

• Check compliance with institutional policy (e.g. do not accept PDFs with passwords)
  
• Check collections for preservation risks (e.g. embedded multimedia content)
  

There are some useful software tools are available, such as:

• qpdf gives detailed information about encryption and password protection
• pdffonts tool that is part of xpdf is useful for checking whether fonts in a PDF are embedded
• The professional version of Adobe Acrobat has a PDF/A validator built into its Preflight tool
• PDF/A validator that is part of the open-source Apache PDFBox library
• VeraPDF has the potential to develop into a full-fledged PDF validator

"The PDF/A standards are nothing more than a set of profiles that impose some restrictions on a PDF, ruling out features that are not well-suited to long-term accessibility."  These features are encryption, non-embedded fonts, multimedia content, and so on. Several tools exist that compare a PDF against PDF/A and report any deviations. These PDF/A validators are typically used to verify PDF/A files but can also be used to detect user-specified risky features in regular PDFs. It is possible to automatically evaluate PDFs against a user-defined set of features. But it is important to check the file because a PDF may satisfy all requirements of PDF/A, and still be broken.

Related posts:

• Ensuring long-term access: PDF validation with JHOVE?    
• PDF/A, PDF for Long-term Preservation
• Preserving the Data Explosion: Using PDF - 25 April 2008
• New ISO standard will ensure long life for PDF documents - 7 October 2005

Digital Preservation Matters.

One downside to digital innovation: as formats die, we lose our past

One downside to digital innovation: as formats die, we lose our past. Jess Zimmerman. The Guardian. 5 August 2015.
     Flash, a proprietary animation software made by Adobe, used to be a leading platform for multimedia but it has fallen out of favor due to security and compatibility issues.  The death of Flash may destroy Flash content by making it not only obsolete but irretrievable. When formats become obsolete often something’s lost in the changeover. "That’s not an inevitable factor of age – books, an unusually obsolescence-resistant format, have remained accessible for hundreds of years. But for many other technologies, continued survival means shedding the past." This has happened to audio and video before, such as with VHS tapes.  "Access to obsolete video formats will always be constrained by the fact that they require an older, tricky-to-source piece of hardware."

"But as early users move into middle age and beyond, we can’t expect our youth – digital or otherwise – to be accessible forever. We’re aging, but the internet’s aging too." We may need to migrate the digital content to other locations to be more shareable and compatible. "But changes in look and outlook don’t erase the past; it remains as a monument to an obsolete age. Changes in technology sometimes do."

Related posts:

• Why Media Preservation Can’t Wait: the Gathering Storm
• Enjoy your digital films and videos while you can... before they disappear
• Video Games and the Curse of Retro
• Meeting the Challenge of Media Preservation: Strategies and Solutions.

Digital Preservation Matters.

DMPonline: recent updates and new functionality

DMPonline: recent updates and new functionality. Sarah Jones. DCC curation webinars. 27 May 2015. [Video and slides].
     This webinar provides an overview of the new functionality in the UK DMPonline tool for data management plans and the development road-map through January 2016. This can be customized and internationalized. The user group aims to make it easier for the community to feed in ideas and direct the development. There is also a reference to the DMP guidance, tools and resources page which is very useful. It contains:

• DMPonline: A web-based tool to assist users to create personalised plans  
• Funders’ data plan requirements: Summary of funders' expectations for data management and sharing plans
• Checklist for a Data Management Plan: Useful questions and guidance for writing data management and sharing plans
• DMP checklist leaflet: A fold-out summary of the Checklist 
• FAQ on Data Management Plans: A short list of key questions pertaining to Data Management Plans
• How to Develop a Data Management and Sharing Plan: A guide on how to meet UK funder expectations for DMPs 
• Guidance and examples: Advice to help you write your data management and sharing plan

Related posts:

• Data Management Plans
• Plan for Increasing Access to Scientific Publications and Digital Scientific Data from NIH Funded Scientific Research
• Where Should You Keep Your Data?
• Sowing the seed: Incentives and Motivations for Sharing Research Data, a researcher's perspective
• AHRQ Public Access to Federally Funded Research

Digital Preservation Matters.

Mapping the Digital Galaxy: The Keepers Registry Expands its Tool Kit

Mapping the Digital Galaxy: The Keepers Registry Expands its Tool Kit. Erin Engle.  The Signal. Library of Congress. August 5, 2015.
     The Keepers Registry released a new version of its website with a suite of new features to help  monitor the archival status of e-journals. The Keepers Registry enables librarians and information managers to discover which journals and continuing resources are being looked after, by whom and how.Before this there was no systematic way with easily accessible information about what is happening for individual journals and issues and volumes, not just serial titles. This is important when archiving digital content from serials and other continuing resources.
 
Related posts:

• Scoping study for a registry of electronic journals. 07 March 2008
• IIPC has released a web archives registry - February 16, 2010

Digital Preservation Matters.

Blu-ray Disc Association to Commence Licensing of Ultra HD Blu-ray

Blu-ray Disc Association to Commence Licensing of Ultra HD Blu-ray. Press release. August 6, 2015.
The Blu-ray Disc Association will start licensing the Ultra HD format August 24, 2015. The format will enable consistent and reliable delivery of Ultra HD content. Ultra HD Blu-ray players will also be required to play back current HD Blu-ray Discs.

Digital Preservation Matters.

Where Should You Keep Your Data?

Where Should You Keep Your Data? Karen M. Markin. The Chronicle of Higher Education. June 23, 2015.
     Federal funding agencies have made it clear that grant proposals must include plans for sharing research data with other scientists. What has not been clear is how and where researchers should store their data, which can range from sensitive personal medical information to enormous troves of satellite imagery.  Although data-sharing requirements have been in place for years, universities have been slow to assist principal investigators make that happen. Now if you don’t comply with the new policies, you might be prohibited from receiving additional grant money. Funding can be withheld from researchers who don’t comply. Principal investigators are urged to place their data in existing publicly accessible repositories and the NIH has a list of repositories. The NSF directs researchers to specific repositories.
The "DMP Tool," hosted by the University of California, provides a free, interactive form that walks you through the preparation of a data-management plan for more than a dozen organizations.

Many libraries are playing a role in this effort and researchers should check with reference librarians for help on this. Data storage and preparation can get complicated and it’s useful to have someone to guide you through the process. Federal agencies plan to establish standards for these so-called "metadata."

Related posts:

• Plan for Increasing Access to Scientific Publications and Digital Scientific Data from NIH Funded Scientific Research
• Data Management Plans
• AHRQ Public Access to Federally Funded Research
• Got Data? A Guide to Data Preservation in the Information Age

Digital Preservation Matters.

Preserving Our Digital Heritage

Preserving Our Digital Heritage. Barry M. Lunt, et al. IEEE. 2008.
     An early paper discussing the research in creating the M-Disc. Current storage technologies available for digital data: magnetic, optical, and solid state, have relatively short lifetimes, ranging from 5 to 30 years. Organizations aware of this problem store multiple copies of the data in multiple places, a strategy very useful where focused attention can be given to the data. But the world of consumer storage is characterized by a store and ignore mentality. The research they have been pursuing is investigate optically appropriate and chemically inert materials, then alter the material using laser irradiation, which leaves a permanent mark with high optical contrast. The recording layer is next to an adhesion layer (it is not an adhesive layer) which is a material which inherently adheres extremely well to polycarbonate. "The layers in our optical storage disc are made from materials that have withstood the test of time and are known to be capable of enduring over 1,000 years."


Related posts:

• How Long Is Long-Term Data Storage? 
• Now available: 100 GB capacity M-Disc
• Dataliths vs. the digital dark age
• Millenniata Announces Results of ISO/IEC 10995 Standard Tests
• Start-up to release 'stone-like' optical disc that lasts forever

Digital Preservation Matters.

Big Web data, small focus: An ethnosemiotic approach to culturally themed selective Web archiving

Big Web data, small focus: An ethnosemiotic approach to culturally themed selective Web archiving. Saskia Huc-Hephe. Big Data & Society. July 2015.

     The fundamental purpose of a Web archive is to retain a version of the fragile and ephemeral digital material found on the Internet for posterity, thereby providing a lasting record of Web objects deemed to be of intellectual and cultural value to current and future generations. A web archive collects only material found on the Internet, safeguarding it from future obsolescence as the internet changes. It is not a record of data, nor a copy, but an entity composed of digital material brought together in a  more restricted environment. 

The article talks about curating a smaller, thematically selected web collection as part of a larger web archive, effectively an archive within an archive. An example is looking at the French community in London. The individual people may not see themselves as belonging to such a community, which questions the validity of constructing a ‘community’ web archive. So the archiving efforts are more creation than curation. One definition of digital curation is ‘maintaining, and adding value to, a trusted body of digital information for current and future use: in other words, it is the active management and appraisal of digital information over its entire life-cycle’. This definition may not fit with web archiving. The curator of a themed web collection is not necessarily a specialist in archival cataloguing or curation, has deep insider knowledge of the ‘field’ for which the collection has been created.

"The significance of the small-scale, micro-Web-archiving approach foregrounded lies in its deployment as a strategy for overcoming the ‘data deluge’ inevitably triggered by non-selective, catch-all repositories."

Digital Preservation Matters.

figshare gets serious about storage; partnership with DuraCloud

figshare gets serious about storage. Dan Valen. Website. June 17, 2015. 
     A new partnership has been created with DuraCloud to better manage and store an institution’s research data. figshare is a repository where users can make their research outputs available in a citable, shareable and discoverable manner. This is combined with the storage, management, and preservation functionalities of DuraCloud. The figshare and DuraCloud collaboration will then ensure all of the captured data is managed and preserved appropriately to align with institutional and funder policies.

The data can be pushed from figshare to DuraCloud that has archiving options which will ensure that the data follows data management plan and grant requirements for archiving and preservation. DuraCloud is open source software and can function within Amazon S3 storage or part of any existing institutional infrastructure. DuraCloud can also connect to Amazon Glacier, Digital Preservation Network (DPN) and Chronopolis. This can be a service for added archiving and preservation of all figshare content. 

Related posts:

• Announcing the ArchivesDirect Price Drop
• New DuraCloud Digital Archiving and Preservation Services.
• DuraCloud Now Offers Low Cost Glacier Storage!
• Chronopolis and DuraCloud: Doing integration right
• Digital Preservation Network (DPN) Launches Member Content Pilot

Digital Preservation Matters.

The Personal Digital Archiving 2015 Conference

The Personal Digital Archiving 2015 Conference. Mike Ashenfelder. Library of Congress Signal. August 3, 2015.   
     The conference is about preserving digital collections outside the scope of large cultural institutions, including family history, community history, genealogy and digital humanities.  Digital video is important as evidence and cultural records, especially of news events. Every individual, family and community should consider the cultural importance of their digital photos.

• “The value of the artifact – and what we keep trying to tell young people – is that they are the authors of a history in the making.” 
• The archives people are creating are "exactly the same kinds of images that filmmakers like us use to make a documentary. People in the future will be looking through their images to try to understand who we are today.” 
• "Personal photos mingle with family personal photos to become a larger archive, a family archive. Facebook has spawned a “local history” phenomena, where members of a community post their personal photos and comments, and the individual personal contributions congeal organically into a community history site."
• Increasingly we hear from colleges and universities, usually — though not exclusively — from their librarians, expressing concern that students and faculty may not be aware of the need to preserve their digital stuff.

Instead of commercial products at the conference, presentations, workshops and posters shared practical information about projects that used open-source tools:
A few of the many presentations at the conference include:

• Activating the Family Photograph Album for the Digital Age
• Photos die, unless we talk about them
• Artists and vendors: Models for Collaboration in the Digital Preservation of Audiovisual Materials
• Does BitTorrent + Private Trackers = The New Film Archive? 
• 5.25 inch floppy disks 
• An Exploration of the Potential Impact of Wearable Computing Technologies on Digital Archiving and Preservation 
• How Online Sociality and Free Terabytes are Shaping Personal Digital Archiving 
• “Yeah, I do think it's very important that identifying information be stored with the file. The big problem is that without that - too much context is lost.”
• “I feel that this is the greatest thing [Social Media Platforms] to happen to photography since the first camera was created."

 

Digital Preservation Matters.

Information Governance Offers a Strategic Approach for Healthcare

Information Governance Offers a Strategic Approach for Healthcare. Lesley Kadlec, et al. Journal of AHIMA. October 2014.
     Information governance is emerging as a strategy that helps organizations achieve their goals. Successful organizations realize that information is a valuable asset and must be carefully managed throughout the information life cycle. The purpose of information governance is two-fold:

1. Stewardship of information 
2. Using information to achieve organization goals

The American Health Information Management Association defines information governance as: "The enterprise-wide framework for managing information throughout its lifecycle and supporting the organization’s strategy, operations, regulatory, legal, risk and environmental requirements." Information governance is an enterprise-wide program that requires leadership and support from the organization’s executive leaders. This is vital in order to focus on assessing risks, evaluating needs, and putting together policies, procedures, and tools. Since retention and information storage is very costly to manage, it is important to manage the information effectively to eliminate redundancy and information that is no longer needed.Sustainability is the key to long term success. Good planning and vision are needed to start a program. Stakeholders need to be involved, as well as ongoing communication, training, auditing, and reporting.

Related posts:

• Information Governance: Why Digital Preservation Should Be a Part of Your IG Strategy
• Managing Chaos Through Digital Governance
• AHRQ Public Access to Federally Funded Research

Digital Preservation Matters.

NARA Bulletin 2015-02: Guidance on Managing Electronic Messages

NARA Bulletin 2015-02: Guidance on Managing Electronic Messages. National Archives. July 29, 2015.
     Bulletin from the National Archives to the heads federal agencies proving records management guidance for electronic messages, specifically for text messaging, chat/instant messaging, messaging functionality in social media tools or applications, voice messaging, and similar forms of electronic messaging systems. There are a wide variety of systems and tools that create electronic messages, and this bulletin is to help develop strategies for managing those electronic messages. Electronic messages created or received in the course of business are records and like all records, they must be scheduled for disposition.
There are challenges with managing these types of messages, which should be met by:

• Develop policies on electronic messages that address some of the challenges 
• Update policies when new tools are deployed
• Provide appropriate tools for employees to manage their work
• Determine a strategy to manage and capture content created in those systems
• Train employees to identify and capture electronic messages
• Use third-party services to capture messages
• Ensure electronic messages, metadata and attachments can be exported from the original system for long term preservation.
• Create a retention guideline for electronic messages to meet business, audit, and access needs
• Personal accounts should only be used in exceptional circumstances. 
• Provide clear instructions to all employees on their responsibility to capture electronic messages created or received in personal accounts

Related posts: 

•  Australian electronic books to be preserved under new laws 
• WebPreserver - Collect Web & Social Media as Legally Admissible Evidence 
• The Many Uses of Rhizome’s New Social Media Preservation Tool. 
• Bill targets messy e-records - May 9, 2008 

Digital Preservation Matters.

The ABCs of Digital Preservation

The ABCs of Digital Preservation. Jenny Mullins. Preservation Services at Dartmouth College. July 21, 2015.
     The purpose of the presentation was to introduce some basic digital preservation concepts, such as choosing file formats, file naming best practices, and the basics of preservation metadata. Also discussed were  tools and models for managing digital materials. The slides of the talk are available. “File extensions tell computers how to read, open and render files. If a file extension is wrong or missing, your computer will be confused. Digital Preservation is about not confusing your computer.”

Formats: Choose open, widely supported file formats:

• Images: .tif, .png
• Text: .pdf, .rtf, .txt
• Audio: .wav
• Video: .avi, MPEG-2 or 4
• Spreadsheets: .csv, .ods

File Naming: Provide unique descriptive names; be consistent.  Only use letters, number, underscores, periods. Include version number or date.

Identify and Describe the content:

• Provenance: How was it created, and by whom? How and why has it changed?
• Context: How does this file relate to others? What is needed to understand it?
• Descriptive: Who? What? Where? Why? When?
• Technical: What are the file characteristics?
• Rights: Who owns it? Who’s allowed to use it and for what purpose?

Some general resources...

• North Carolina Department of Cultural Resource's Digital Preservation Best Practices and Guidelines
• North East Document Conservation Center resources  
• Keeping Personal Digital Photographs 
• Resources for Embedding metadata in images
• FADGI: Technical Guidelines for Digitizing Cultural Heritage Materials
• FADGI: Digital File Formats for Digital Tape Reformatting
• Activist's Guide to Archiving Video 
• Digital Curation Centre: How-to Guides & Checklists
• Library of Congress: Sustainability of Digital Formats

There are many software tools and training materials available for large scale data preservation.

Digital Preservation Matters.

Managing Chaos Through Digital Governance

Managing Chaos Through Digital Governance. Patrick K. Burke, Lisa Welchman. CIO Insight. March 23, 2015.
     An interview that discusses digital governance and change. Organizations should have a digital leadership who, with the executive level, should initiate a digital strategy. If the digital strategy does not tie in to the organizational goals then problems result. Digital strategies that start at the bottom of an organization and are built only on a foundation of digital best practices may be helpful but may not accomplish what the organization needs.  Organizations need to make sure that digital area experts make decisions about digital standards after getting input from key stakeholders. It is the stakeholders that know what is supposed to be happening.

• Chaos has a way of slipping into an organization and creating problems that grow exponentially over time. 
• With so much innovation, collaboration and bottom-line focused initiatives involving digital assets, there is a very real need to provide a framework to manage this change. Digital governance helps with this process.  
• The opposite of digital governance is making it up and funding it as you go along, maintaining a project instead of operational view when it comes to digital. 
• The golden rule is if you know what you are making, how you are making it and who is supposed to be making it, then you ought to be governing firmly, particularly if you are trying to bring something to scale. 
• If you are just exploring, then some chaos can be good. That’s why I often recommend that digital teams maintain a research and development component that gets to explore the chaos.

Digital Preservation Matters.

Software dependent content

Software dependent content. Euan Cochrane. Digital Continuity Blog. August 5, 2015.
     People assume that emulation is only for preserving the “look and feel” of digital objects. Instead, it can be used to interact with content that requires a particular and limited range of software environments. Software can add, remove, or alter content, not just the “look and feel”. The post introduces the term “software dependent content” to refer to digital content that is to be interacted with, rendered, viewed or consumed. "I use this term to refer to content that requires a particular and limited range of software environments in order to be interacted with, rendered, viewed or consumed." It is useful when discussing the need to invest in emulation based digital preservation approaches.

Digital Preservation Matters.

Alfresco Brings the New York Philharmonic’s Digital History to Life

Alfresco Brings the New York Philharmonic’s Digital History to Life. Press release. Business Wire. July 29, 2015.
     The New York Philharmonic is using the Alfresco enterprise content management system for the Leon Levy Digital Archives, which will contain more than 3 million pages of correspondence, operation files, marked scores, and other materials. The project consists of two phases:

1. Document restoration. All the Philharmonic’s paper-based content will be imaged and ingested into the system. 
2. Digitize nearly 10,000 hours of Philharmonic audio and video files, storing the audio and rights information, and making them available in an online archive.

The Archives is doing this to make material broadly and freely available and to have a copy for preservation.  The Philharmonic plans to develop partnerships with academic institutions and music conservatories to create curriculum that will focus on material available in the Digital Archives, which will hold about 5 TB of data by close of the current phase.

Digital Preservation Matters.