How State CIOs Should Preserve Digital Records -- Electronic records are at risk and vulnerable

How State CIOs Should Preserve Digital Records. Phil Goldstein. November 05, 2018.
     States are not well prepared for long-term preservation of digital records, which means the electronic records are at risk and vulnerable. State governments are living in the world of digital records, which has many challenges with preserving content. The records are essential for state governments and must be preserved 

• Electronic records require attention to ensure they are preserved and accessible. They are more complex to preserve than paper records
• “Sustained attention and resources are needed to ensure the long-term management and accessibility of our nation’s electronic records.”
• Collaboration is key, since digital records management involve multiple organizations.  “Collaborative effort is key to developing and adopting best practices and sustainable models for the long-term preservation of electronic records,”
• “adequate employee awareness and training activities are keys to ensuring that employees correctly carry out new or existing policies and procedures and understand how to use any new technologies associated with improved electronic records management.”
•  “Establishing fixity, or the property of a digital file or object being fixed or unchanged, is a critical part of confirming evidentiary status of electronic records,”
• some systems for document management don’t preserve the content, structure, context and integrity of the record over time.  “States must select technologies that properly manage and store electronic records, while ensuring that the inevitable obsolescence of the technology does not compromise the records’ integrity or accessibility,” 
• “The need for digital preservation of state electronic records will outlast commercial service providers and current technological infrastructures. The state needs to clearly understand its rights regarding its data and how the preservation provider is helping it perform its obligations to its citizens.”
• State CIOs can help state archives and records management personnel perform a cost-benefit analysis about outsourcing preservation services in relation to data security, the report says. 
• Contracts with third-party digital preservation service providers should “establish responsibility for functions that are critical to ensuring the integrity of state data including fixity checking and audits or compliance with state government legal responsibilities,” according to the report. 
• State CIOs and archivists should also establish audit trails when working with a third-party preservation service provider. “A verifiable audit trail of the activities involved in the processing of digital records ensures that the reliability and authenticity of the data is secure,” the report says. 

Digital Preservation Matters.

Electronic Records Management Guidelines: Long-Term Preservation

Electronic Records Management Guidelines: Long-Term Preservation. March 2012, Version 5. Minnesota Historical Society. [Guidelines - Contents]

     Over the course of time, organizations generate many records. Some are of short term duration while others are to be kept permanently. “Tools such as migration, conversion, metadata, and eXtensible Markup Language (XML) will help you not only preserve your records, but also realize their full value.” Records need to be preserved, since “the greatest possible access to certain government information and data is essential to allow citizens to participate fully in a democratic system of government.”   

Some key concepts presented in the section on Long-Term Preservation:

1. Needs Assessment. Understanding the value of the records and the information they contain will help guide decisions, determine their retention requirements, the access and use of the records, as well as preservation options. 
2. Physical Storage Options. Record access requirements will help determine the type of storage to use, specifically
1. Online storage. Immediately available on the network
2. Near-line storage.  Records are stored in automated optical disk or tapes libraries attached to a network.
3. Offline storage.  Records are stored on removable media that must be retrieved manually.
3. File Format Options. For long-term file preservation, non-proprietary formats are preferred, but they also have limitations.
4. Digital Preservation Techniques. There are several approaches to ensure that electronic records remain useful over time.
1. Emulation. Using emulator programs to simulate the behavior, of original programs.
2. Encapsulation. Combining the object to be preserved with all of the necessary details of how to interpret it within a wrapper or package.
3. Migration. This is the more common approach, which is the process of  moving files to new media or computer systems to maintain their use. 
5. Preservation Planning.  “A preservation plan should address an institution’s overall preservation goals and provide a framework that defines the methods used to reach those goals.  At a minimum, the plan should define the collections covered by the plan, list the requirements of the records, practices and standards that are being followed, documentation of policies and procedures related to preservation activities, and staff responsibility for each preservation action.” This plan needs constant updating and cost/benefits must be addressed. Policies should be developed to put the plan into practice.

Digital Preservation Matters.

Electronic Records Task Force Phase 2 Final Report

Electronic Records Task Force Phase 2 Final Report. John Butler, et al. University of Minnesota. August 23, 2017. [PDF, 68 pp.]
     The University of Minnesota Libraries sponsored an Electronic Records Task Force to monitor established workflows and to develop new workflows, policies, procedures and mechanisms for processing and providing access to electronic records. They are focused on the development of processing activities, best practices and guidelines. Creating finding aids, which are published online through ArchivesSpace, are the first step in providing access to electronic records. The long-term preservation of electronic records is a concern and this effort continues to be a work-in-progress. To keep up with the influx of electronic records, the Electronic Records Task Force provides the following recommendations:

1. Staffing: Hire a permanent full time employee to work exclusively with electronic records
2. Long-term Management: Create an Electronic Records Management Group to address ongoing electronic records needs
3. Preservation: Review current workflows and long-term management requirements to address immediate and long-term solutions for file backup, recovery, and preservation according to policies and standards
4. Security: Conduct a thorough review of security requirements
5. Equipment: Establish initial and ongoing financial support for hardware, software and collections
6. Access to Materials: Explore options for providing access to electronic records, including both access and preservation of these materials.

Project Tasks and Deliverables

1. Develop Workflows for Processing Ingested Collections
2. Define Processing Levels (minimal, intermediate, full)
3. Develop Access Methods that Address End-user Needs, Copyright, Data Privacy and other Information Security Requirements
4. Monitor Ingest Workflows and adjust as necessary

Additional notes:

• "In the long-term, a full-time dedicated staff person is the most responsible approach to working effectively and efficiently, to achieve quality work, and to maintain our leadership role in the field of electronic records management. This is arguably the only way to address the ingest and processing activities that assist with long-term access to and preservation of electronic materials. Without a dedicated person who has an in-depth understanding of evolving workflows and protocols and who can provide a consistent approach with curatorial staff, any headway in addressing the records being collected will be made slowly."
• The goal of processing unique electronic archival material is to make it available to end users, whether they be skilled researchers or a high school student working on a project.
• Given divergent requirements, a singular asset management, backup, and preservation solution may not be a feasible goal in either the near or long term. However, efforts can be made to establish a limited number of processes to manage the vast majority of preservation use cases.

Digital Preservation Matters.

Universal Electronic Records Management Requirements

Universal Electronic Records Management Requirements. Courtney Anderson. National Archives Records Express. August 4, 2017.
     The National Archives has released the Universal Electronic Records Management Requirements as part of the Federal Electronic Records Modernization Initiative (FERMI). Universal ERM Requirements identify high level business needs for managing electronic records. The program requirements are derived from existing NARA regulations, policy, and guidance and are a starting point for agencies to use when developing system requirements. "Records management staff should work with acquisitions and IT personnel to tailor any final system requirements". The document contains an abstract, a glossary, and lists of lifecycle requirements and transfer format requirements.
There are six sections based on the lifecycle of electronic records management:

1.    Capture
2.    Maintenance and Use
3.    Disposal
4.    Transfer
5.    Metadata
6.    Reporting

The requirements are either “program” requirements, relating to the design and implementation of policies and procedures, or “system” requirements, providing technical guidance for creating or acquiring ERM tools, which also indicate “Must Have” or “Should Have”. NARA will be supporting these requirements going forward and will be updating them to stay current with changes in technology, regulations and guidance products.

Digital Preservation Matters.

From digital dark age to digital enlightenment

From digital dark age to digital enlightenment.  Caroline Pegden. National Archives. 17 February 2016.
     Recent media reports have talked about the ‘digital Dark Age‘.  This is a major challenge, now and for the years to come for institutions in the archives sector, who are concerned with managing, preserving and providing access to born-digital records. This is important for the UK National Archives because some government departments will soon transfer born-digital records to The National Archives under the Public Records Act. As the National Archives has been working on how to do this, their philosophy has been ‘learning by doing’. They have reviewed what other archival institutions around the world are doing to manage digital records and have been testing the process of transfers "to design and test the new process to appraise, select, sensitivity review, transfer, preserve and give access to born-digital records." Two major challenges are:

1. extracting meaning from unstructured digital record collections in order to make appraisal and selection decisions.
2. sensitivity reviewing born-digital records at scale without having to read all the individual documents

Most government departments’ information is on unstructured shared drives; some departments had up to 190 terabytes of information in email servers.  Technology-assisted-review, a process using reviewers and a combination of computer software and tools to electronically classify records may have interesting applications for the archives sector. "Although there is no ‘silver bullet’ or completely automated solution, technology-assisted review offers ways to prioritise and reduce the information to be manually reviewed."  Two reports are available that highlight challenges and shows how technology-assisted review could help addressing some of these challenges. 

1. The digital landscape in government 2014-15: business intelligence review
2. The application of technology-assisted review to born-digital records transfer, Inquiries and beyond: research report

Digital Preservation Matters.

Risk management guide for the secure disposal of electronic records

Secure destruction of electronic records. Archives New Zealand. 2 October 2015.
     Blog post on the secure and complete destruction of electronic records plus all copies and backups. Destruction of paper records are mostly straightforward. However it is not so easy to confidently delete electronic records. The processes to destroy digital records should be secure, irreversible, planned, documented and verifiable.  The article has examples of risks of not destroying records,  as well as resources on how to implement the destruction records. In addition there is a new guide on the benefits of disposal and the risks of not disposing of records: Risk management guide for disposal of records.

[Disposal and destruction of digital records may not seem like it has anything to do with digital preservation, but it is an important part of records management. More than just that, it can be a needed part of the submission and ingest processes made multiple copies of sensitive content have been created before or while adding the content.  Or if you have been given media to add and then must dispose of the media afterwards. -cle]

Digital Preservation Matters.

NARA Bulletin 2015-02: Guidance on Managing Electronic Messages

NARA Bulletin 2015-02: Guidance on Managing Electronic Messages. National Archives. July 29, 2015.
     Bulletin from the National Archives to the heads federal agencies proving records management guidance for electronic messages, specifically for text messaging, chat/instant messaging, messaging functionality in social media tools or applications, voice messaging, and similar forms of electronic messaging systems. There are a wide variety of systems and tools that create electronic messages, and this bulletin is to help develop strategies for managing those electronic messages. Electronic messages created or received in the course of business are records and like all records, they must be scheduled for disposition.
There are challenges with managing these types of messages, which should be met by:

• Develop policies on electronic messages that address some of the challenges 
• Update policies when new tools are deployed
• Provide appropriate tools for employees to manage their work
• Determine a strategy to manage and capture content created in those systems
• Train employees to identify and capture electronic messages
• Use third-party services to capture messages
• Ensure electronic messages, metadata and attachments can be exported from the original system for long term preservation.
• Create a retention guideline for electronic messages to meet business, audit, and access needs
• Personal accounts should only be used in exceptional circumstances. 
• Provide clear instructions to all employees on their responsibility to capture electronic messages created or received in personal accounts

Related posts: 

•  Australian electronic books to be preserved under new laws 
• WebPreserver - Collect Web & Social Media as Legally Admissible Evidence 
• The Many Uses of Rhizome’s New Social Media Preservation Tool. 
• Bill targets messy e-records - May 9, 2008 

Digital Preservation Matters.

National Archives kicks off 'born-digital' transfer

National Archives kicks off 'born-digital' transfer. Mark Say. UKAuthority. 24 June 2015.
The National Archives is looking at the long term issue of keeping records accessible as the technology in which they are originally created changes.

"To make sure born-digital records can be permanently preserved we’re engaged in what we call parsimonious presentation, in which we’re making sure it can be used by the next trends of technology being developed. We want them to be easily viewed in 10 years’ time, although we cannot plan for 100 years as there’s no way we can know what the technology will look like."

“To ensure records will still be used in the same way we want to see what the technology is going to do in the next 10 years.

“Digital preservation is a major international challenge. Digital technology is changing what it means to be an archive and we are responding to these changes.

“These records demonstrate how we are leading the archive sector in embracing the challenges of storing digital information for future generations. We are ensuring that we are ready to keep the nation’s public records safe and accessible for the future, whatever their format.”

Digital Preservation Matters.