International Federation of Film Archives: Survey on Long-term Digital Storage and Preservation

Digital Statement PartV Survey on Long-term Digital Storage and Preservation. Céline Ruivo and Anne Gant. FIAF Technical Commission, International Federation of Film Archives. April 2019.

    The sustainability of digital files and formats for long-term preservation has been a major concern in this field for almost two decades now. The FIAF Digital Preservation Principles, published in 2016, use the OAIS (Open Archival Information System) Model. Increasingly, film archives are publishing their own technical specifications online. Digitizing a film includes not only archiving a final result (the master), but also archiving the “raw files” which are uncompressed. Some of the results of the survey of 16 institutions who responded:

• DPX is the main format used for preservation: 14 archives
• TIFF is used as a second preservation format: 4 archives
• Most use 4K resolution when they scan 35mm negatives for preservation 
• Few have written technical specifications for the deposit of new digital acquisitions, which are mostly born-digital films.
• Some archives use lossless compression for long-term preservation of a master to reduce storage space 
• Some archives are considering implementing the FFV1 format this year for storing files. 
• A checksum called framemd5 is integrated with the files MKV/FFV1.
• The recording back to film of restorations is applied by 8 archives

Sound
    In terms of sound, digital formats are more variable than image formats, depending upon their final distribution (cinema or TV broadcast). RAW formats are usually the same as the restored files.

• Most of the archives use a tape system for long-term conservation. 
• They generally wait for 2 generations to migrate their data to reduce the cost 
• Access storage is by a server that allows direct access to the files.
• Most of the archives store and manage their files in their own facility. 

Conclusion
     This initial survey of the current digital landscape shows there is much more work to be done to get a global view of digital film archiving, and to hear from more archives at all stages in the development of digital workflows. Some conclusions that can be drawn from the current set of responses:

1. There is a stabilization in language and a conceptual clarity emerging about the stages of a digital workflow within archives. The terms are becoming clear and are recognized as necessary parts of daily archival practice. This will allow for better information exchange and better comparison of workflows.
2. There are some choices which seem to be predominant, such as 10 bit Log DPX, for example, or the use of ProRes, LTO, .wav files, etc. It is helpful to detail the reasons why certain archives chose uncommon formats or processes.
3. There are reasons behind each archive’s choices, which make sense at the given moment. But it would be useful to revisit this survey in 5-10 years (or sooner), and see how digital film practices and archiving are progressing.

See also:

• Digital Preservation Principles v2 [PDF] 
• FIAF: Resources of the Technical Commission

Digital Preservation Matters.

Next Phase OAIS Review

Next Phase OAIS Review. Barbara Sierman. Digital Preservation Seeds. March 24, 2019.
     The OAIS standard had its 5-year review in 2017, which resulted in over 200 suggestions for change. All the changes were discussed in the DAI Working Group and and most of them were accepted. The next step is for the updated OAIS standard to go to CCSDS and ISO for final approval. The main part of the changes concerned terminology, both clarification and consistency.

Some concepts got a more extensive description, while others have changed. The new OAIS standard shows that a transparent process can lead to a standard that reflects the current practices. The standards group will now have a final opportunity to decide whether all suggested changes are clear and implementable.

Digital Preservation Matters.

Digital preservation is a mature concept, but we need to pitch it better

Digital preservation is a mature concept, but we need to pitch it better. Dave Gerrard. Digital Preservation at Oxford and Cambridge.  6 December, 2016.
     The OAIS standard can be confusing for newcomers to the field, and one of the potentially confusing areas is the Administrative area. It looks "like a place where much of the hard-to-model, human stuff had been separated from the technical, tool-based parts." The diagram is busier and more information-packed than other areas, and thus could use more modeling. The standard may be easier to use if there were other documents focusing on the ‘technical’ and ‘human’ aspects.

Communication, particularly an explanation to funders, about the importance of digital preservation is vital. It will help to have an 'elevator pitch' to explain simply what digital preservation is. The post suggests "Digital Preservation means sourcing computer-based material that is worthy of preservation, getting that material under control, and then maintaining the usefulness of that material, forever." [Some of these words may be easily misunderstood.]

The "OAIS standard is confusing" "but it has reached a level of maturity: it’s clear how much deep thought and expertise underpins it."  The digital preservation community is ready to take their ideas to a wider audience: "we perhaps just need to pitch them a little better".

Digital Preservation Matters.

OAIS: a cage or a guide?

OAIS: a cage or a guide? Barbara Sierman. Digital Preservation Seeds. December 3, 2016.    
     Post about the OAIS standard and asking if it is a restriction or a guide. OAIS, the functional model, the data model and metrics in OAIS and the related standards like the audit and certification standard. "OAIS is out there for 20 years and we cannot imagine where digital preservation would be, without this standard." It is helpful for discussing preservation by naming the related functions and meta data groups. But it lacks a link to implementation and application for daily activities. OAIS is a lot of common sense put into a standard. The audit and certification standard, ISO 16363, is meant to explain how compliance can be achieved, a more practical approach.

Many organisations are using this standard to answer to the question "Am I doing it right?" People working with digital preservation want to know the approach that others are using, the issues that they have solved. The preservation community needs to "evaluate regularly whether the standards they are using are still relevant in the changing environment" and a continuous debate is required to do this. In addition, we need evidence that practical implementations that follow OAIS are the best way to do digital preservation. Proof of what worked and what did not work is needed in order to adapt standards, and the DPC OAIS community wiki has been set up to gather thoughts related to the practical implementation of OAIS and to provide practical information about the preservation standards,

Digital Preservation Matters.

Selection and Appraisal in the OAIS Model

Selection and Appraisal in the OAIS Model. Ed Pinsent. DART Blog. 7 September 2016.
     The post asks if the OAIS Model accommodate the skills of selection and appraisal, then suggests that it cannot.  The Model presents an over-simplified view where in a state that is all ready to preserve, which ignores the beginning processes.There is a need to define the pre-ingest stage in OAIS, but there needs to be  a greater recognition of the archivists' Selection and Appraisal skills, can have tremendous value in digital preservation. Archivists assess the value of the content in a contextual framework, based on other records in the archive and in the context of provenance. It requires an understanding of context, provenance, record series, to help identify the potential value of content. A Series model is the "foundation for all Archival arrangement, and is the cornerstone of our profession". It is difficult to see where the record / archival series is in all this.  "The integrity and contextual meaning of a collection is being overlooked, in favour of this atomised digital-object view.

OAIS, if strictly interpreted, could bypass the Series altogether in favour of an assembly line workflow that simply processes one digital object after another."  The blog post asserts that there is a need to rediscover the value of Appraisal and Selection and its importance in the digital realm. 

Digital Preservation Matters.

Minimal Effort Ingest

Minimal Effort Ingest. Bolette Ammitzbøll Jurik, Asger Askov Blekinge, Kåre Fiedler Christiansen. Statsbiblioteket, Danmark. March 29, 2016. Poster   Abstract
     The poster won best poster award at iPres2015. An expensive part of ingesting digital collections into repositories is the quality assurance, which traditionally happens before ingest.  This ensures that only data which complies with the repository data formatting and documentation standards is preserved.

With Minimal Effort Ingest, which is a different approach to ingest and Quality Assurance, the data is ingested as it is; quality assurance happens after ingest. This method makes it possible to ingest the content quickly, especially older collections. Quality assurance failures are handled within the repository. Preservation actions can be taken in the future as needed and as resources are available. "Repositories implementing Minimal Effort Ingest are eventually consistent, content- and preservation-wise, with the OAIS model."

"Performing preservation actions post-ingest on the repository content, rather than during ingest provides benefits in both development effort and preservation liability."

Digital Preservation Matters.

Free OAIS Beginners Course – Update

Free OAIS Beginners Course – Update. Stephanie Taylor. DART Blog, University of London Computer Centre. 9 December 2015.
     An online course ‘A Beginners Guide to the OAIS Reference Model’ was launched in November for those interested in learning more about OAIS. The course remains open and free to anyone interested. "It’s been fantastic to see so much international engagement. We’ve also had a great cross-section of students in many roles from many kinds of organisations, including national memory institutions, higher education, cultural heritage, national and local government departments and the commercial sector." The blog has the link to sign up for the course.

Digital Preservation Matters.

DPC invites members to review the OAIS Standard

DPC invites members to review the OAIS Standard. William Kilbride, et al. Digital Preservation Coalition,  Open Preservation Foundation. October 21, 2015.
"DPC is delighted to welcome members to participate in the review of OAIS, work that will hold our interest for a couple of years and which we aim to build into a platform for collaboration among our diverse members in the future.

The OAIS standard published by both the Consultative Committee for Space Data Systems (CCSDS) and as ISO14721 has been highly influential in the development of digital preservation. As a reference model it provides a common basis for aligning disparate practice in diverse institutional settings. A range of standards have emerged around and related to OAIS including PREMIS (for preservation metadata), ISO16363 (for certification) and PAIMAS (for exchange between Producers and Archives).

Since OAIS was initially proposed the digital preservation community has grown tremendously in absolute numbers and in diversity. OAIS adoption has expanded far beyond the space data community to include cultural heritage, research data centers, commerce, industry and government.

The digital preservation community has – we have! – a responsibility to keep our standards relevant. The upcoming ISO review of the OAIS standard in 2017 offers a chance for a cooperative, transparent review process. It also creates an opportunity for further community building around OAIS and related initiatives.

"The outcome from this activity is not simply a wiki nor is it a set of recommendations. By providing a shared open platform for the community that gathers around the OAIS we aim to ensure on-going dialogue about our standards and their implementation in the future.
In this sense the 2017 review is a milestone on the way to an engaged and empowered community rather than a destination.

• OAIS Community forum via a wiki: Your feedback and the discussions on this wiki will provide raw material for an editorial committee of the most active participants to formulate recommendations which will result in a formal submission to the 2017 review. So sign in and add your views!
• Exploring official mechanisms: Official mechanisms for the review of ISO standards are well established via National Standard Bodies and these will be explored and used to give input for the review.
• Active Interaction: Ensuring inclusion for this large, diverse community will mean collaborative virtual meetings are necessary but we all recognize the value of meeting face to face and will seek to enable this.

Join the community and contribute your views on the wiki here: http://wiki.dpconline.org/index.php?title=OAIS_Community

Digital Preservation Matters.

Digital Archives & Preservation Systems

Digital Archives & Preservation Systems. Kari R. Smith.  MIT Institute Archives. Engineering the Future of the Past Blog. July 1, 2015. [PDF]
     Very good presentation on fitting tools, processes and systems together. These open source tools are being developed and used in the digital archives and digital preservation communities to address the needs and gaps in systems for meeting standards, such as OAIS, PAIMAS, and Trustworthy Digital Repositories. They enable description, discovery, delivery, and preservation of digital collections. Some of the tools mentioned in the presentation:

• ArchivesSpace: archives collection management and discovery
• Archivematica: digital preservation system (processes files chosen for preservation and creates Archival Information Packages and Dissemination Information Packages)
• Access to Memory (atom): archives and special collections discovery and delivery system and collections management system
• BitCurator and BitCurator Access: digital forensics tools and processes for archival and special collections material
• ePADD: electronic mail processing, access, discovery system
• Fixity: enables the preservation actions of fixity checking of files in archival (or other) storage. Includes reporting on file attendance and file integrity.
• QuickView Plus: render the files so you can appraise them and make decisions
• Binder: integrates with Archivematica to view administrative, technical, descriptive, and preservation metadata related to AIPs and the relationships among the metadata
• Webrecorder.io: Manual capture of sites, creates WARC file

Lifecycle Environment

• Storage Pathways
•   Steps in Manage Digital workflow showing Storage Areas and Activities
•    Submission Storage -> Ingest Storage -> Archival Storage -> Dissemination Storage
• Digital Content Management Workflow
• Diagram showing steps in the Manage Digital Workflow and the Storage Areas.
• External Content Creators
• Archives Collections Security Requirements
• Archives Management Systems provide support for the archival workflow, including appraisal, accessioning, description, arrangement, publication of finding aids, collection management, and preservation.”

Digital Preservation workflows may involve the integration of several tools, such as:

• ArchivesSpace – Archivematica – ArchivesSpace
• ArchivesSpace - atom – Archivematica – atom
• BitCurator – ArchivesSpace – Archivematica - ArchivesSpace

Monitor digital files with Fixity and Binder. Look at mapping standards and tools to your local situation. Evaluate how these meet your Mission, Tasks, Components:

• What is our mission and goals? 
• Break them into tasks to accomplish
• Determine where they happen during workflow
• Identify
• Information inputs
• Digital content inputs
• Intended output
• Decide how these fit with the lifecycle storage areas
• Select tools that assist us accomplish our goals then Monitor, Adjust, Repeat

Digital Preservation Matters.

Beyond TIFF and JPEG2000: PDF/A as an OAIS Submission Information Package Container

Beyond TIFF and JPEG2000: PDF/A as an OAIS Submission Information Package Container. Yan Han. Library Hi Tech. 2015.
     PDF/A can be used as a file format, but it can also be used as OAIS SIP containers. The PDF/A open standards can "simplify digitization process, reduce digitization cost, improve production substantially and build more confidence for preservation and access." PDF/A can be used as an Archival Information Package container.

The three main goals of PDF/A are to:

• provide a way to present the appearance of documents independent of the tools and systems used
• provide a framework for recording the context and history of electronic documents in the metadata
• define a framework for representing the logical structure of electronic documents within conforming files

A typical SIP may consist of a directory containing the following information"

• Content: 
• Preservation master files (such as TIFF images files). 
• Access files (such as a PDF or JPG / JPG2000 files).
• Other content (such as OCR data).
• Preservation description: 
• Preservation metadata in the TIFF header
• Other structural and technical metadata
• Checksum files.
• Packaging information: 
• Directory and File naming, structural metadata.
• Descriptive information: 
• Descriptive metadata saved in digital management system, catalog, or textual/XML files.

"The key requirement of PDF/A is that it is self-described and self-contained so that it can bereproduced exactly the same way with different software in various platforms." It will include all information needed to display the content in the PDF/A file (text, images, fonts, and color profiles).

Master file formats should be non-proprietary, open and documented international standards that are  commonly used. The files should be unencrypted, and should be uncompressed or else use lossless compression. The author of the article recommends using PDF/A as the preferred file format for text and image files, and possibly using it as an OAIS SIP container. The author shows how PDF/A is a better file format than the currently preferred TIFF or JPEG2000 formats.

There are several issues with PDF/A naming and implementation. The most critical need is reliable open source software for producing and validating PDF/A files.

Digital Preservation Matters.

Information Governance: Why Digital Preservation Should Be a Part of Your IG Strategy

Information Governance: Why Digital Preservation Should Be a Part of Your IG Strategy. Robert Smallwood. AIIM Community. July 6, 2015.
     The post looks at Information Governance and digital preservation. The post author wrote the first textbook on information governance (IG). He used key models as part of this, such as the  Information Governance Reference Model (IGRM), E-discovery Reference Model and the OAIS model.  The question to answer is whether or not long term digital preservation should be a part of a information governance strategy.

Information Governance is defined as: 

a set of multi-disciplinary structures, policies, procedures, processes and controls to manage information at an enterprise level that supports an organization's current and future regulatory, legal, risk, environmental and operational requirements. 

• "Long term digital preservation applies to digital records that organizations need to retain for more than 10 years."
• digital preservation decisions need to be made early in the records lifecycle, ideally before creation.
• Digital preservation becomes more important as repositories grow and age.

"The decisions governing these long term records - such as digital preservation budget allocation, file formats, metadata retained, storage medium and storage environment - need to be made well in advance of the process of archiving and preserving."

"All this data - these records - cannot simply be stored with existing magnetic disk drives. They have moving parts that wear out. The disk drives will eventually corrupt data fields, destroy sectors, break down, and fail. You can continue to replace these disk drives or move to more durable media to properly maintain a trusted repository of digital information over the long term."

If you move to a cloud provider that makes preservation decisions for you, then "you must have a strategy for testing and auditing, and refreshing media to reduce error rates, and, in the future, migrating to newer, more reliable and technologically-advanced media."

Your information governance strategy is incomplete if do not have a digital preservation strategy as well. Your organization "will not be well-prepared to meet future business challenges".

Digital Preservation Matters.

Filling the Digital Preservation Gap. A Jisc Research Data Spring project. Phase One report - July 2015

Filling the Digital Preservation Gap. A Jisc Research Data Spring project. Phase One report - July 2015. Jenny Mitcham, et al. Jisc Report. 14 July 2015.
     Research data is a valuable institutional asset and should be treated accordingly. This data is often unique and irreplaceable. It needs to be kept to validate or verify conclusions recorded in publications. Preservation of the data in a usable form may be required by the research funders, publishers, or  universities. The research data should be preserved  and available for others to consult  after the project that generated it is complete.This means the research data needs to be actively managed and curated. "Digital preservation is not just about implementing a good archival storage system or ‘preserving the bits’ it is about working within the framework set out by international standards (for example the Open Archival Information System) and taking steps to increase the chances of enabling meaningful re-use in the future."

Accessing research data is clearly already a problem for researchers when formats and media become obsolete. A 2013 survey showed that 25% of respondents had encountered the “Inability to read files in old software formats on old media or because of expired software licences”. A digital preservation program should address these issues. Open Archival Information System and it uses standards such as PREMIS and METS to store metadata about the objects that are being preserved.  A digital preservation system, such as Archivematica recommended in the report, would consist of a variety of different systems performing different functions within the workflow. "Archivematica should not be seen as a magic bullet. It does not guarantee that data will be preserved in a re-usable state into the future. It can only be as good as digital preservation theory and practice is currently and digital preservation itself is not a fully solved problem."

Research data is particularly challenging from a preservation point of view because of the many data types and formats, many of which are not formats that digital preservation tools and policies exist for, thus they will not receive as a high a level of curation when ingested into Archivematica.
The rights metadata within Archivematica may not fit the granularity that would be required for research data. This information would need to be held elsewhere within the infrastructure.

The value of research data can be subjective and difficult to assess and there may be disagreement on the value of the data. However, the bottom line is "in order to comply with funder mandates, publisher requirements and institutional policies, some data will need to be retained even if the researchers do not believe anyone will ever consult it." Knowing the types of formats used is a key to digital archiving and planning, and without that there will be problems later. In the OAIS Reference Model, information about file formats needs to be part of the ‘Representation Information’ that an end user must have to open and view a file.

Digital Preservation Matters.

Digital Preservation 101, or, How to Keep Bits for Centuries

Digital Preservation 101, or, How to Keep Bits for Centuries. Julie C. Swierczek. scholar.harvard.edu website. June 4, 2015.
An interesting presentation that looks at what digital archivists need to preserve electronic records permanently. It discusses:

• OAIS model for long-term digital preservation;
• requirements of a trustworthy digital repository;
• preferred file formats for long-term storage; 
• digital forensics and FRED machines;
• definitions and differences of “archives” and “backup copies”.

Digital Preservation Matters.

Data Archives and Digital Preservation

Data Archives and Digital Preservation. Council of European Social Science Data Archives. June 1, 2015.
Data Archives and Digital Preservation Data archives play a central role in research. Data is considered “the new gold”. There is increasing pressure on researchers to manage, archive, and share their datadata archives. It is important to securely store research data, and to allow researchers to reuse data in their own analyses or teaching.

Archives are much more than just a storage facility; they actively curate and preserve research data. They must have suitable strategies, policies, and procedures to maintain the usability, understandability and authenticity of the data. There are also numerous requirements from users, data producers, and funders. In the social science research data preservation and sharing, archives have the added responsibility of protecting the human subjects of the research.

The CESSDA site has many resources. Some of these are:

• What is digital preservation 
• OAIS 
• Data appraisal and ingest 
• Documentation and metadata 
• Access and reuse 
• Trusted digital repositories: audit and certification.

Digital Preservation Matters.

What Do We Mean by ‘Preserving Digital Information’? Towards Sound Conceptual Foundations for Digital Stewardship

What Do We Mean by ‘Preserving Digital Information’? Towards Sound Conceptual Foundations for Digital Stewardship. Simone Sacchi. Dissertation University of Illinois at Urbana-Champaign.2015. [PDF]
Preserving digital information is a fundamental concept in digital and data stewardship. This dissertation explains what successfully ‘preserving information’ really is, and provides a framework for understanding when and why failures might happen and how to avoid them. The lack of a formal analysis of digital preservation is problematic. Some notes and quotes from the dissertation:

• At a high level of generality, bit preservation means enabling the possibility for the same (set of ) bit sequence(s) to be discriminated at different points in time, and, potentially, across changes in the underlying storage technology." 
• Bit level preservation is a mean, not the goal, in digital stewardship. 
• As suggested by the OAIS definition of digital preservation, successful digital preservation is about “maintaining” or “preserving” information.
• Preserving information appears to be a metaphorical expression where a complex set of requirements needs to be satisfied in order for an agent to be presented with intended information
• The best contemporary theories of digital preservation do not focus on the preservation of any sort of object, but rather on preserving access.
• it is impossible to preserve a digital document as a physical object. One can only
  preserve the ability to reproduce the document.
• "You cannot prove that you have preserved the object until you have re–created it in some form that is appropriate for human use or for computer system applications.”
• “digital records are not stable artefacts”; they last only when certain circumstances are met
• Bit preservation is only the first required step for successful digital stewardship. Interpreting the bits such that an intended digital material obtains through appropriate performances is essential as well.
• Successful digital preservation of information can be conceived as sustained and reliable communication mediated by digital technology and agents involved in the communication process.

Digital Preservation Matters.

A Digital Preservation Environment Maturity Matrix for NSLA Libraries

A Digital Preservation Environment Maturity Matrix for NSLA Libraries. Sarah Slade, David Pearson, Libor Coufal. iPres Proceeduings. October 2014.
The National and State Libraries of Australasia (NSLA) established a Digital Preservation Group to understand the state of digital preservation in the various libraries and to determine the core requirements for managing the preservation of digital collections. They listed and described the functional components of an ideal digital preservation environment and created a matrix of the current stage of development against each component for each NSLA library. Related projects by others include the National Digital Stewardship Alliance and BenchmarkDP.

NSLA Digital Preservation Environment Maturity Matrix

• Underlying Assumptions
• actively collecting digital material
• committed to preserving its digital materials for the long term.
• staff (or vendor) dedicated to the project
• intends to comply with OAIS

• Functional Component

An ideal digital preservation environment should contain a mix of policies, processes and resources (including staff and technologies). The OAIS model calls for organizations to:

• Negotiate for and accept information from information producers.
• Obtain sufficient control of the information for long-term preservation.
• Determine the designated user community.
• Ensure the information is independently understandable to the designated community without the need of special resources.
• Follow documented preservation policies and procedures.
• Make the information available to the designated community.

Instead of just listing the functions, they created a set of questions about the OAIS functionality in order to help responders describe their organization's level of digital preservation maturity, and included the OAIS functions:

• Pre-ingest Activities
• Ingest
• Archival Storage
• Data Management
• Administration
• Digital Preservation Planning
• Access

• Maturity Model

The Group modified the Capability Maturity Model, using 5 levels:

1. Initial. Processes are usually ad hoc. Achievement depends on the competence of the people in the organization and not on the use of proven processes. Products and services usually exceed budget and schedule
2.  Repeatable. Basic digital preservation processes are established.  Digital preservation achievements are repeatable, though not all activities.
3. Defined. Digital preservation activities are performed and managed according to documented plans. Processes for digital preservation are established and improved over time. 
4. Managed. Management can effectively control the digital preservation effort, using precise measurements.  Quantitative quality goal for digital preservation processes
5. Optimising. Organisation focuses on continually improving process performance. The effects of deployed digital preservation process improvements are measured and evaluated against the quantitative process-improvement objectives.   

"NSLA has identified digital preservation as an area of priority. The importance of this area to NSLA libraries is reflected in the creation of the Digital Preservation Group and its support of the Group’s work to date. The results from the Digital Preservation Environment Maturity Matrix reveal that NSLA libraries are on the right path but have some way to go before digital preservation processes are mature, sustainable and fit for purpose. Collaboration on policies, products and infrastructure will continue to address these needs."

Digital Preservation Matters.

Trustworthiness: Self-assessment of an Institutional Repository against ISO 16363-2012

Trustworthiness: Self-assessment of an Institutional Repository against ISO 16363-2012. Bernadette Houghton. D-Lib Magazine. March/April 2015.
Digital preservation is a relatively young field, but progress has been made for developing tools and standards to better support preservation efforts. There is increased interest in standards for the audit and certification of digital repositories because researchers want to know they can trust digital repositories. Digital preservation is a long-term issue. The Trustworthy Repositories Audit and Certification (TRAC) checklist has been widely used as the basis of the activities. It later became ISO 16363 (based on the OAIS model) which contains 105 criteria in 3 areas:

1. Organizational infrastructure (governance, structure and viability, staffing, accountability, policies, financial sustainability and legal issues)
2. Digital object management (acquisition and ingest of content, preservation planning and procedures, information management and access)
3. Infrastructure and security risk management (technical infrastructure and security issues)

 "Undertaking a self-assessment against ISO 16363 is not a trivial task, and is likely to be beyond the ability of smaller repositories to manage." An audit is an arms-length review of the repository, requiring evidence of compliance and testing to see that the repository is functioning as a Trusted Digital Repository.  Most repositories at this time are in an ad hoc, still-evolving situation. That is appropriate at this time, but a more mature approach should be taken in the future. The assessment process would rate features for: Full Compliance, Part Compliance, Not Compliant. The conclusions in the article include:

• Self-assessment is time-consuming and resource-heavy, but a beneficial exercise
• Self-assessment is needed before considering external certification. 
• Certification is expensive.
• Get senior management on board. Their support is essential.
• Consider doing an assessment first against NDSA Levels of Digital Preservation  
• Repository software may be OAIS-compliant, but it doesn't mean your repository is also
• Not all ISO 16363 criteria have the same importance. Assess each criteria accordingly
• ISO 16363 is based on a conceptual model and may not fit your exact situation
• Determine in advance how deep the assessment will go.
• Document the self-assessment from the start on a wiki and record your findings  

Digital Preservation Matters.

ArchivesDirect hosted service

ArchivesDirect website. February 18, 2015.
ArchivesDirect is a web based hosted service of Archivematica offered by DuraSpace for creating OAIS-based digital preservation workflows with content packages that are archived with DuraCloud and Amazon Glacier. It includes open source preservation tools, and generates archival packets using microservices, PREMIS, and mets xml files. ArchivesDirect is intended for small to mid sized institutions. Duraspace is a partnership with DSpace, Fedora, and Vivo.

Pricing and subscription plans include:
ArchivesDirect Standard (System, training, 1 TB): $11,900
ArchivesDirect Digital Preservation Assessment: $4,500
Additional Storage in Amazon S3 and Glacier: $1,000/TB/year

Digital Preservation Matters.

Digital Preservation Coalition publishes ‘OAIS Introductory Guide (2nd Edition)’ Technology Watch Report

Digital Preservation Coalition publishes ‘OAIS Introductory Guide (2nd Edition)’ Technology Watch Report. Brian Lavoie.  Digital Preservation Coalition. Watch Report. October, 2014. [PDF]

The report describes the OAIS, its core principles and functional elements, as well as the information model which support long-term preservation, access and understandability of data. The OAIS reference model was approved in 2002 and revised and updated in 2012. Perhaps “the most important achievement of the OAIS is that it has become almost universally accepted as the lingua franca of digital preservation”.

The central concept in the reference model is that of an open archival information system. An OAIS-type archive must meet a set of six minimum responsibilities to do with the ingest, preservation, and dissemination of archived materials: Ingest, Archival Storage, Data Management, Preservation Planning, Access, and Administration. There are also Common Services, which consist of basic computing and networking resources.

An OAIS-type archive references three types of entities: Management, Producer, and Consumer, which includes the Designated Community: consumers expected to independently understand the archived information in the form in which it is preserved and made available by the OAIS. This is a  framework to encourage dialogue and collaboration among participants in standards-building activities, as well as identifying areas most likely to benefit from standards development.

An OAIS-type archive is expected to:

• Negotiate for and accept appropriate information from information producers;
• Obtain sufficient control of the information in order to meet long-term preservation objectives;
• Determine the scope of the archive’s user community;
• Ensure the preserved information is independently understandable to the user community
• Follow documented policies and procedures to ensure the information is preserved against all reasonable contingencies
• Make the preserved information available to the user community, and enable dissemination of authenticated

An OAIS should be committed to making the contents of its archival store available to its intended user community, through access mechanisms and services which support users’ needs and requirements. Such requirements may include preferred medium, access channels, and any access restrictions should be clearly documented.

 The OAIS information model is built around the concept of an information package, which includes: the Submission Information Package, the Archival Information Package, and the Dissemination Information Package. Preservation requires metadata to support and document the OAIS’s preservation processes, called Preservation Description Information, which ‘is specifically focused on describing the past and present states of the Content Information, ensuring that it is uniquely identifiable, and ensuring it has not been unknowingly altered’. The information consists of:

• Reference Information (identifiers)
• Context Information (describes relationships among information and objects)
• Provenance Information (history of the content over time)
• Fixity Information (verifying authenticity)
• Access Rights Information (conditions or restrictions)

OAIS is a model and not an implementation. It does not address system architectures, storage or processing technologies, database design, computing platforms, or other technical details of setting up a functioning archival system. But it has been used as a foundation or starting point. Efforts, such as TRAC, have been made to put the attributes of a trusted digital archive into a ‘checklist’ that could be used to support a certification process. PREMIS is a preservation metadata initiative that has emerged as the de facto standard. METS, and XML based  document form, has become widely used for encoding OAIS archival information packages.

The ‘OAIS reference model provides a solid theoretical basis for digital preservation efforts, though theory and practice can sometimes have an uneasy fit.’

Digital Preservation Matters.

ARMA 2014: The Convergence of Records Management and Digital Preservation

ARMA 2014: The Convergence of Records Management and Digital Preservation. Howard Loos, Chris Erickson. October 2014. [PDF]
Presentation on records management and digital preservation given at the ARMA 2014 conference.
Notes:

• Records Management mission: To assist departments in fulfilling their responsibility to identify and manage records and information in accordance with legal, regulatory, and operational requirements
• RIM Life Cycle to DP Life Cycle
• Challenges and successful approaches
• Storing records permanently with M-Discs
• Introduction to Digital Preservation, challenges, format sustainability, media obsolescence, metadata, organizational challenges,
• Life of digital media
• Best practices and processes
• OAIS model
• Rosetta Digital Preservation System
• Library of Congress Digital Preservation Outreach & Education (DPOE) Network

Digital Preservation Matters.