Preservation of AV Materials in Manuscript Collections. Training for AV format identification and risk assessment. Actions to take

Preservation of AV Materials in Manuscript Collections; Internal Training.  Ben Harry. Brigham Young University. November 2018.
     The presentation is not yet available on the internet. Some notes from the training:
“There is now consensus among audiovisual archives internationally that we will not be able to support large-scale digitisation of magnetic media in the very near future. Tape that is not digitised by 2025 will in most cases be lost.”  -NFSA.gov.au, Oct. 2018

The problem with AV is Fragility:

• Playback equipment is disappearing
• Knowledgeable experts are disappearing
• Materials breaking down
• Untrained handling easily destroys materials

The solution to the fragility is to address materials in a timely manner:

• Priority and Speed and Efficiency
• Train transfer operators
• Untrained handling easily destroys materials

A Challenge of AV is Neglect:

• Unable to describe AV Content adequately in finding aids or catalogs. 
• Requires certain level of specific knowledge of formats and physical carriers.
• Requires machine to read information that may not be available
• Time-consuming process for little reward
• Expensive, unstructured, uncoordinated

To overcome the challenge:

• Digitize material for description in basic processing
• Time-consuming process for little reward
• MUST be a lean process to minimize the effect upon processing

Audio-video preservation requires a certain level of specific knowledge. Staff must be trained to recognize and report AV Formats. Also, it is important to have risk assessment guidelines to help make informed decisions. Coordinate efforts and resources to reduce confusion, prioritize and set goals, unify our proposals for equipment and man power.


Actions to take:

• Prioritize Formats for Migration / Reformatting
• Maintain Transparent Records on Preservation and Access
• Link Preservation and Access (one does not happen without the other)
• Provide Curators with AV Assessment tools
• Organize a Queue System to keep things equitable (what about 12 items per month, per curator? Adjust as Necessary)
• Create Digital File Naming guidelines
• Establish Access and Preservation format standards for AV materials:

 For Access and Preservation, the following standards will be used:

Audio Preservation

• Preservation Format:  PCM / wav 96 kHZ sampling   24-bit depth. 1 GB/Hour
• Access Copy: mp3.  Music: 256 kbps. Voice: 192 Kbps.

Video Preservation: Standard Def

• Preservation Format: ffv1 / mkv 720 x 486. 33 GB/Hour  
• Access Copy: H.264 / mp4

Video Preservation: Hi Def

• Preservation Format: ffv1 / mkv Native: 1080i / 1080p. 100 GB/Hour?  
• Access Copy: H.264 / mp4

Film Preservation

• Preservation Format: RGB ffv1 / mkv 1080i scan (MPS capability ceiling). 100 GB/Hour?  
• Access Copy: H.264 / mp4

Archive and delivery methods:

• Preservation: Rosetta
• Access: various options are available. 

Digital Preservation Matters.

Preservation of AV Materials in Manuscript Collections. Training for AV format identification and risk assessment

Preservation of AV Materials in Manuscript Collections; Internal Training.  Ben Harry. Brigham Young University. November 2018.
     Ben Harry, Curator of Audiovisual Materials and Media Arts History at Brigham Young University, provided some internal training concerning AV format identification and risk assessment. Here are some assessment tools for AV materials.

Digital Preservation Matters.

Assessing and Quantifying Risk to Digital Media Materials

Assessing and Quantifying Risk to Digital Media Materials. Lance Thomas Stuchell. Bits and Pieces. August 31, 2016.
     A post written by Sarah Breen, Alix Norton, and Alexa Hagen. Archives are increasingly facing challenges in preserving digital media materials; creating digital processing workflows and workstations is one often discussed challenge. This article discusses a framework for assessing risk of loss to digital archival materials and shows that the methodology can highlight materials most susceptible to loss. This will help administrators demonstrate the need for immediate intervention and processing.

The methodology used a formula for calculating risk to physical collections:

"The formula yields a calculation of the magnitude of a given risk (MR) by multiplying the factors of the fraction of the collection that is susceptible (FS), loss of value (LV), probability of risk (P), and extent of the risk (E). By giving each of these factors a value between 0 and 1, we calculated MR values for the overall magnitude of a variety of risks, also between 0 and 1. While this formula is often used to assess risks over a 100 year period, due to the nature of the short lifespan and rapid obsolescence of digital media, we have used this formula to assess risks over a 10 year period".

External risks would affect the collection as a whole, and would include fire, theft, damage, and lack of funding to continue preservation projects. Internal risks are more specific to the physical digital media format, such as obsolescence of format and media degradation. Management, funding, administrative decisions and the storage environment can also be areas of high risk.

The highest risks assessed include:

1. degradation and obsolescence, 
2. lack of funding, and 
3. potential loss of management support. 

The article recommends actions be taken to mitigate these risks early by:

• migrating digital content to a stable content management system, 
• lowering relative humidity of the storage environment, 
• securing the lowest cost digital storage option that remains aligned with the library’s policy, and 
• advocate to library and university administration showing the need for preservation

These recommendations should significantly reduce the highest risks and help ensure the preservation of the digital information.

Digital Preservation Matters.

Exploring appraisal, quality assurance and risk assessment in the data continuum

Exploring appraisal, quality assurance and risk assessment in the data continuum.   Linda Ligios. Pericles blog. 8 March 2016.
     PERICLES presented a workshop on "appraisal, quality assurance and risk assessment in relation to the lives of complex digital objects."   It introduced the key concepts of :

• model-driven preservation in a continually evolving environment
• appraisal processes that lend themselves to being automated,
• development plans for tools on appraisal, risk assessment and quality assurance 

Three main dimensions

1. Risk – probability of an entity being non-usable
2. Proximity – time frame in which we consider risk/impact
3. Impact – potential loss of functionality and cost of mitigating actions

"Policies should always reflect the vision of the institution and therefore contain principles that are more aspirational in nature."  The  PERICLES model-driven preservation approach:



Related topics:

• PERICLES: towards model-driven preservation. Poster by Jean-Yves Vion-Dury.
• PERICLES Environment Extraction Project 

Digital Preservation Matters.

Applying DP Standards For Assessment & Planning

Applying DP Standards For Assessment & Planning. Bertram Lyons. PASIG 2016. March, 2016.
     ISO 16363:2012. Audit & Certification of Trustworthy Digital Repositories defines recommended practices for assessing the trustworthiness of digital repositories. The document will help those who audit repositories, but also those to design or redesign their digital repository processes. Some highlights from the standard:

3.1 Governance and Organizational viability: The repository shall have a collection policy or other document that specifies the type of information it will preserve, retain, manage, and provide access to. Without the policy the collection scope is unclear and it becomes difficult to say no to out of scope content. The standard expects a policy to exist and be documented.

4.2 Ingest: Creation of AIPs: Organizations should have a description of how AIPs are constructed from SIPs. It should document all changes to the processes, as well as defining what happens to the content (such as normalization of files, etc.)

5.2 Security Risk Management: The repository should have a written disaster preparedness and recovery plan, including at least one off-site backup of all preserved information together with an off-site copy of the recovery plan. This means the organization should be prepared administratively.

The elements are scored as follows

• 0 - non-compliant or not started
• 1 - slightly compliant (needs a lot of work to do in address the requirement.
• 2 - half compliant: partially addressed but still significant work to do
• 3 - mostly compliant: mostly addressed and working on full compliance.
• 4 - fully compliant: can demonstrate the requirement is comprehensively addressed.

Elements needed:

• Documentation: records of policy, procedure, and outcomes of activities
• Policy: the definition of approaches and protocol for repository functions and procedures
• Procedures: specification of preservation and infrastructure management activities
• Software: development or configuration of preservation systems
• Infrastructure: procurement, monitoring, and management of hardware infrastructure
• Organization: organizational infrastructure including funding, staffing, and strategy
• Action Plan

Digital Preservation Matters.

A multi-agent approach for autonomous digital preservation

A multi-agent approach for autonomous digital preservation. J. Pellegrino,  M. Maggiora, W. Allasia. Multimedia & Expo Workshops, 2015 IEEE International Conference. June 29 2015-July 3 2015. 
     Digital obsolescence is caused by the ongoing development of new software and new formats, so the risk of obsolescence can be estimated from a global environment.The model described in the article
presents two main strategies to cope with digital obsolescence: migration and emulation, and then focuses on migration. Migration consists in converting digital objects into a new and more recent format and is the process we are going to focus on in this work.

An agent can be defined as a computer system that is capable of autonomous actions to meet its  objectives. The environment and all the agents that interact together to share information constitute a multi-agent system. The agents interact together in a variety of ways to meet the objectives. An agent-based model could be set up to deal with digital preservation issues; they can acquire, evaluate and share information in order to understand obsolescence risks and the best preservation action to perform. This will require a trust relationship between archive entities. There are some tools that perform obsolescence identification and metadata extraction, such as AONS (Automated Obsolescence Notification System), DROID (Digital Record and Object Identification), and JHOVE (JSTOR/Harvard Object Validation Environment).

Preservation processes require institutions to define a preservation plan. PLANETS (Preservation and Long-term Access through Networked Services) and Scout (developed within the SCAPE Project) are tools that help with identifying preservation issues and managing digital repositories. None of the tools discussed so far use the described model which aims to emulate a distributed environment where  archive entities can share information to find solutions to the digital preservation issues.

The model is self updating from preservation actions taken, and is made up of three main parts:

1. Global: it includes all those variables accessible to every agent. 
2. Entities: contains the declarations of all the species of agents that take part in the model. 
3. Experiment: dedicated to the experimental setup.

The work presented in this paper provides a novel approach to the decision processes concerning one of the most common digital preservation issues such as the migration process. The agents in this migration process have the capability of communicating, cooperating and propagating information about the performed actions in order to help each of the other agents find the best solution to a given preservation issue. The goal is to provide a framework so users can simulate different digital preservation scenarios and approaches. The model has show the stability of the framework under various use cases.

Digital Preservation Matters.

Risk Assessment as a Tool in the Conservation of Software-based Artworks

Risk Assessment as a Tool in the Conservation of Software-based Artworks. Patricia Falcao.
   The article looks at the use of risk assessment methodologies to identify and evaluate vulnerabilities of software-based artworks. Software-based art is dependent on technology. Two consequences of this:
1. Because electronic equipment is usually mass-produced, there are very few cases where one individual device is essential for the artwork.
2. On the other hand, when the equipment is no longer commercially available it becomes very difficult to replace any of its elements.
A sculpture conservator may be able to re-make a missing element for a sculpture by using the same or similar materials but a time-based media conservator cannot always re-make obsolete electronics. A particular artwork may use custom-made software. "Any software, in turn, usually requires a specific operating system. All the programs, from the firmware to the operating system, must run properly. All settings, plug-ins, and codecs must be in place. Without all of these, there is no artwork."

This means that each artwork is a custom-made system; the components may vary with each iteration of the work and as technology changes. A conservator understand "how these components are used in the particular system and how they influence the risks and options for long-term preservation." With the conservation of contemporary art,  obsolescence only affects an artwork once something stops working. But the effect of obsolescence will increase over time. 

Software-based artworks are similar to time-based media, bu they are more vulnerable to those risks because:

1. Systems are customized for each artwork.
2. Systems are easily changed, so that connecting a archival computer to the Internet could cause it to run an automatic update that causes the file will no longer run. 
3. The technical environment is rapidly changing.

The degree of significance can be evaluated by

1. Provenance 
2. Rarity or representativeness 
3. Condition or completeness 
4. Interpretive capacity

A procedure for the acquisition of software-based artworks being developed is composed of simple actions during acquisition that can diminish the impact of obsolescence in the medium-term. It is important to discuss the artwork, technology, and possible preservation measures with the artists and technical staff.  The conservator should identify and define:

1. The display or presentation parameters 
2. What can or cannot be changed, and within what limits. 
3. Identify obsolescent elements and create a plan for recovery.  
4. How the artist wants the artwork preserved. Identify core elements and migration strategy.  
5. Understand the system (hardware, software). Test the system with the artist and staff. 

Over the lifetime of the artwork,

1. Document the system and any changes over time  
2. Prevent changes such as automatic updates
3. Monitor obsolescence issues with the components of the work.  
4. Re-evaluate preservation needs regularly. 

Some steps that can be taken to reduce the failure and obsolescence

1. Make clones of the computer’s hard drive immediately upon acquisition. 
2. Create an exhibition copy of the system, possibly with the artist and staff. 
3. Gather operation manuals, service manuals, and hardware specifications. 
4. Save the software versions, source code, libraries, and programming tools necessary to read project files. 

For long term preservation,

1. Continue to implement the preservation strategies identified.
2. Develop clear procedures for the acquisition of software-based artworks. 
3. Identify software tools useful for preservation. 
4. Test recovery strategies and confirm results over time. 
5. Develop relationships with experts in the fields required for preservation. 

Software-based preservation will require more than just the conservator. It will also require help from the technology field and many tools.

Digital Preservation Matters.