Wednesday, January 31, 2018

The Safeguarding of the Audiovisual Heritage: Ethics, Principles and Preservation Strategy

The Safeguarding of the Audiovisual Heritage: Ethics, Principles and Preservation Strategy: IASA-TC 03, Edition 4, 2017. The International Association of Sound and Audiovisual Archives. January 31, 2018.
     These are some notes from the latest revision of a core document for the audiovisual preservation community and provides principles and strategies for audio visual preservation. "The future of preserving digital material for the long term will be one of managing a pathway between the choices we make now and those choices we must make in the future. We must act decisively now even though we know that technological developments will not necessarily align with those choices. Though no choice is a final one, a well informed decision will consider the process for navigating to the new. Major changes in the current revision include a widening of scope to include moving image content, and a greater acknowledgement of the prevalence of file-based digital material alongside its carrier-based equivalent."

Preserving audiovisual material requires completing three related tasks:
  1. Preserving the stability and optimal readability of the physical carrier through best practices.
  2. Maintaining or renewing the technological system required to access the information.
  3. Transferring the information to other sustainably accessible, file-based formats while there is still access to the original information.
  • Audiovisual carriers are generally more vulnerable to loss of information than conventional materials due to damage caused by poor handling, poorly maintained equipment or by poor storage.
  •  market-driven obsolescence of formats means there is a finite window of opportunity for digitally preserving carrier-based content
  • efforts must be made to preserve carriers in useable condition for as long as is feasible.
  • " the preservation of the document in the long term can only be achieved by copying the contents to new carriers/systems while this remains possible."
  • "Separating the primary information from the original carrier raises the question of future authentication of the sound and images."
  • Responsible preservation of digital data requires systems and a technical infrastructure, the monitoring of the condition of files, and the existence of plans for media migration and format migration." These topics are discussed in the Open Archival Information System (OAIS) Reference Model (ISO 14721) and the Trusted Digital Repositories (ISO 16363).
  • "It is strongly recommended that metadata be written according to established standards, in as consistent a fashion as possible"
  • Generally, priority should be given to those documents that are at greatest risk, through either degradation or technical obsolescence
  • "The archive must, therefore, keep itself and its employees updated with the latest scientific and technical information from the field of audiovisual archiving. This will include information concerning the extraction of both primary and secondary information from carriers, and improvements in preservation and restoration practices."

Saturday, December 30, 2017

The 'Bit List' of Digitally Endangered Species

The 'Bit List' of Digitally Endangered Species. Digital Preservation Coalition. November, 2017.
     This list of Digitally Endangered materials is a community effort to discover which digital materials are most at risk, as well as those which are relatively safe thanks to digital preservation. This effort is to raise awareness of the need to preserve digital materials, but also "celebrate great digital preservation endeavors as entries become less of a ‘concern,’ whilst still highlighting the need for efforts to safeguard those still considered ‘critically endangered.’" The Risk Classifications and categories are:
  • Lower Risk: Digital materials that do not meet the requirements for other categories but where there is a distinct preservation requirement.  Failure or removal of the preservation function would result in re-classification to one of the threatened categories.
  • Vulnerable: Digital materials when the technical challenges to preservation are modest but responsibility for care is poorly understood, or where the responsible agencies are not meeting preservation needs.  
  • Endangered: Digital materials that face material technical challenges to preservation or responsibility for care is poorly understood, or where the responsible agencies are poorly equipped to meet preservation needs.  
  • Critically Endangered: Digital materials that face material technical challenges to preservation, there are no agencies responsible for them or those agencies are unwilling or unable to meet preservation needs.  
  • Practically Extinct: Digital materials that are inaccessible by most practical means and methods.  
  • Concern:  Digital materials where there is a preservation concern but they have not yet been assessed by BitList.

Thursday, November 30, 2017

Libraries and the Preservation of Cultural Heritage: Building on the Past to Develop Our Future

Libraries and the Preservation of Cultural Heritage: Building on the Past to Develop Our Future. Chris Erickson. XXVII Congresso Brasileiro de Biblioteconomia, Documentação e Ciência da Informação. Fortaleza, Brazil. October 19, 2017.
     A few weeks ago I had the wonderful opportunity to visit Brazil and give this presentation on digital preservation at the Congresso Brasileiro de Biblioteconomia, Documentação e Ciência da Informação. Many thanks to Adriana Cybele Ferrari (FEBAB) and Anderson de Santana. The presentation provides an overview of digital preservation concepts as well as our approach to preservation and our workflow.

Friday, November 17, 2017

AFI, Library of Congress Celebrate 50 Years of Film Preservation

AFI, Library of Congress Celebrate 50 Years of Film PreservationPenelope Poulou. Voice of America. November 14, 2017.
     The American Film Institute and the Library of Congress are partners in film preservation. The Library of Congress facility has an ongoing preservation process where technicians transfer films onto a sturdier polyester-type film material, which if stored properly can last for centuries. The digital conversion reaches wider audiences on a multi-platform basis, including streaming. "Not only are they archiving these movies, they are also circulating to television channels, television stations.
But the  Nitrate Film Vault manager says digital preservation may be an oxymoron.  “How do you save digital material? 'Cause digital as a rule is very iffy. You have only a couple of different ways you can store it, you can store it magnetically or optically or on a card, but none of those are permanent. Something can disrupt them and the stuff is gone.”

Whether stored in their original format or restored on newer film or digitally, the important thing is that these films are kept for posterity. “Film is a huge part of our history. And, if we don’t cherish it and preserve it, it will not be with us. So, we have to do that.”

Friday, November 10, 2017

Staffing for Effective Digital Preservation 2017: An NDSA Report

Staffing for Effective Digital Preservation 2017: An NDSA Report. Winston Atkins, et al. NDSA. October 2017. 
     This excellent report, based on a recent survey, looks at how organizations staffed and organized their digital preservation functions, and compares it with the survey done in 2012. This is a report worth studying.

Survey respondents were from the following organizations:
  • Academic library or archives (46%)
  • Government entities (11%)
  • Museum (8%)

"Organizations establishing or scaling up digital preservation programs are faced with many staffing, scoping, and organizational decisions. Some of the questions that need to be answered include":
  • How many staff members are needed and what kinds of skills, education, and experience should they have? 
  • What types of positions should the institution create? 
  • Should it hire new staff or retrain existing staff? 
  • What functions should be included in the preservation program, provided by other parts of the organization, outsourced, or implemented through collaboration with other organizations? 
  • What organizational and staffing models work well? 

From the survey, organizations reported: 

  • an average of 13.6 FTE are working in digital preservation activities, but ideally the organizations would double that to 27.5 FTE 
  • there is a need for more digital archivists,software developers, and cataloger/metadata analysts. 
  • 68% of organizations retrained existing staff for at least some digital preservation functions, 
  • 42% of organizations also hired experienced digital preservation specialists. 
  • Staffing for an organization managing 1–50 TB 
    • Current: 10.7 FTE 
    • Ideal: 30.6 FTE

Content amount and Collection growth:
  • 58.6% were preserving 1–50 TB of digital content, 
  • 16.5% were preserving 51–100 TB, 
  • 14.3% were preserving 101–500 TB,
  • 8.3% were preserving more than 500 TB. 
  • 73.2% expected less than 25% growth in the collection.  
  • In 2012, 68% expected up to a 49% growth. 

Preservation activities and organization:
  • Most organizations prefer conducting most digital preservation activities in-house
  • Only 32% of the organizations had a dedicated digital preservation department
  • 46% were not satisfied with how the digital preservation function was organized within their organization
  • 25% believed it was organized properly.
  • Satisfaction decreased from 2012, when 43% agreed or strongly agreed that their digital preservation functions were well-organized.
  • One of the most striking findings was the increased percentage of respondents who reported that they were not satisfied with the way the digital preservation function was organized
  •  52% of respondents participate in at least one consortium or cooperative network. Benefits include:
    • networking (68%), 
    • training (57%), 
    • storage space (54%) 
    • Consulting (35%), 
    • access interface (33%) 
    • communications/marketing (28%), 
    • programming (25%), 
    • federated search (16%)  
  • Department that takes the lead for digital preservation: 
    • Library / Archives 69.1%
    • Information Technology (IT) 16.0%
    • Preservation department 6.2%
    • Other 8.6%

Other general comments of interest about digital preservation staffing issues include:
  • “Continuing education is a must, so that staff can stay up-to-date on current trends and the latest news in technology.” 
  • “It should be organized keeping in mind extra skills of an individual in addition to his/her specialist skill set. 
  • “Staffing is critical to success.... All the pieces must be in place for a successful digital preservation effort.”

Importance of Qualifications for digital preservation staff in 2017, in order:
  1. Knowledge of digital preservation standards/best practices
  2. Communication
  3. Passion and motivation for digital preservation
  4. Collaboration
  5. Analytical skills
  6. Project planning/management

Tuesday, October 31, 2017

Preserving the public record on television is becoming an ever-more-urgent task

The Devolution Will Be Televised. Peter B. Kaufman and Jeff Ubois. The Nation. October 18, 2017.
     Preserving the public record on television is an increasingly critical challenge for the country and the world because it is a primary source that historians and others will rely on to document this administration. Audio and video will be a major part of the public record for this time period. "There is no question that, as we look to the end of this century and how our time will be remembered, we will look back at our news and our culture through moving image and recorded sounds."

Preserving the public audiovisual record on television, and all audiovisual media is urgent task, especially for memory institutions. Moving images are the most popular form of media today: over 80 percent of web traffic is video. Many professionals and organizations are working on this, but strong funding mechanisms are weak or missing. In October 1997, the Library of Congress issued its first report, “Television and Video Preservation 1997” the need for preserving these materials. The American broadcasting records are historical and cultural materials which are "a key to understanding our civilization”.  Many film and audiovisual assets were already being lost due to media degradation and equipment obsolescence.

National strategies are needed for publishing and distributing our digitized and born-digital archival material.  "As the recent scrubbing of government websites has shown, we must rely on non-governmental institutions to help ensure that our archives are never permanently altered to reflect political expediencies. Indeed, we should ensure that the video records of presidential press conferences, banking debates, foreign-policy debates, and all such public activity is preserved and remains accessible to future citizens, journalists, and political figures. We need to recommit to preserving all of our televised triumphs and tragedies."

Saturday, October 14, 2017

Personal Digital Archiving Guide Part 2: Media Types and File Formats

Personal Digital Archiving Guide Part 2: Media Types and File Formats.  Scott David Witmer. Bits and Pieces. August 15, 2017.
     This helpful follow-up post focuses on “born-digital” files created on the computer, and the "characteristics of digital file formats that you should consider when deciding how to preserve your digital materials".  For information on digitizing, it links to guides and handouts, including scanning, recommendations, audio conversion, video conversion, storage, and others.

"The best time to think about preservation is before you create your files." Making decisions early, including organization and metadata, will make it easier to preserve digital files over time.  The post reviews:
  • The trade off between Quality vs. Size of digital files
  • Lossless versus lossy compression
  • File Formats by Media Type
  • Formats for Text, Image, email, audio, video
 Metadata is also important. Be consistent and descriptive when naming or grouping files.

Friday, October 13, 2017

Personal Digital Archiving Guide Part 1: Preservation Planning

Personal Digital Archiving Guide Part 1: Preservation Planning. Scott David Witmer. Bits and Pieces. April 26, 2017.
     Digital materials require active intervention if we want to be able to use them over time. Technology is constantly changing, digital files are at risk because of  accidental deletion or disaster. Having a preservation plan can help avoid data loss. "Do what makes the most sense to you to manage your own digital materials. Even if it’s not practical for you to follow all of these steps, any amount of effort to preserve your digital material is better than none!"
  • Identify: What digital materials do you want to save? 
  • Gather: Where are the digital files you want to keep? Gather all of the files you want to save onto one hard drive. Makes copies of them on other devices. 
  • Select: Decide what you want to keep. 
  • Organize: Know what the files are and where to find them. Descriptive information will help. Give the files meaningful names group files together. 
  • Back-up Storage: After the files are gathered and organized, back them up. Follow the 3-2-1 Rule: Make 3 copies and 2 additional copies of all the files. Use 2 different types of storage media, such as an external hard drive or in cloud storage. Put one of the copies in a different location from the other 2 copies.
  • Check the files periodically to make sure they are still usable, especially right after you back them up.  
  • Update your long-term digital storage to a new storage device every 5–7 years, as significant upgrades in technology occur.

Thursday, October 12, 2017

Five Organizational Stages for Digital Preservation

The Five Organizational Stages of Digital Preservation. Anne R. Kenney & Nancy Y. McGovern. "Digital Libraries: A Vision for the 21st Century..." 2003.
     I have been re-reading this interesting paper in preparation for an upcoming presentation, and realize the great information in it and the opportunity to reflect on where we are and where we are going. Some notes and quotes that I really like:
  • The world is becoming increasingly dependent on digital information.... Despite the increasing evidence documenting the fragility and ubiquity of digital content, cultural repositories have been slow to respond to the need to safeguard digital heritage materials.
  • Of all the preservation challenges facing us, none is more pressing than developing workable solutions to digital preservation.
  • The reason for the lag in institutional response to the problem "lies in the fact that most of the attention given to digital preservation has focused on technology as both the root of the problem and the basis for the solution."
  • The technological methods "that reduce things to on or off status— either you have a solution or you do not. This either/or assessment gives little consideration to the effort required to reach the on stage, to a phased approach for reaching the on stage, or to differences in institutional settings. Nor does it take into account that a partial program at one institution may represent a fully mature program at another."
  • The goal of digital preservation is to maintain the ability to display, retrieve, and use digital material in the face of rapidly changing technological and organizational infrastructures. Unfortunately, there is no single best way to do just that, nor is there agreement on long-term solutions.
  • In this paper, we describe five definable stages that cultural repositories will pass through on their way to developing a fully mature digital preservation program. 
  • Each of these stages is clearly delineated, characterized by key attributes and organizational responses. Some of the stages may be shortened, and an institution may be further advanced in one aspect over another, but they must all be passed through and in the same sequence.
  • The Five Organizational Stages:  The five stages of organizational response to digital preservation are:
    1. Acknowledge: Understanding that digital preservation is a local concern;
    2. Act: Initiating digital preservation projects;
    3. Consolidate: Seguing from projects to programs;
    4. Institutionalize: Incorporating the larger environment; and
    5. Externalize: Embracing inter-institutional collaboration and dependency.
  • Perhaps the most immediately valuable contribution of the Trusted Digital Repository report is the framework of TDR attributes. The six attributes of the TDR framework are: administrative responsibility, organizational viability, financial sustainability, technological and procedural suitability, system security, and procedural accountability. 
  • The report defines the characteristics of each attribute that together address core legal, economic, technical, and other organizational issues, and break what is often presented as the monolithic digital preservation problem into manageable parts. 
  • A notable feature is that technology is not the central focus or first consideration in the framework.
  • Organizational stages for digital preservation have the potential to provide a more effective communication tool, to define a metric for quantifying progress towards a comprehensive digital preservation program, and to establish benchmarks for setting organizational goals.

Wednesday, October 11, 2017

Digital Preservation, Eh?

Digital Preservation, Eh? Alexandra Jokinen. bloggERS! February 14, 2017.    
     This is a post about international perspectives on digital preservation and about digital preservation in an institution in Canada. One way they are working on digital preservation, which they see as a very large, very complex (but exciting!) endeavour is to "start on a small scale, focusing on the processing of digital objects within a single collection, and then using those experiences to create documentation and workflows for different aspects of the digital archives program."  They chose one collection to start with and the first area of focus was appraisal. Their next step will be to physically organize the material, and the final steps will be to take the born-digital content that has been collected and create Archival Information Packages for storage and preservation with Archivematica . They want to "ensure that solid policies and procedures are in place for maintaining a trustworthy digital preservation system in the future."

Monday, October 09, 2017

Cultural Heritage and Digital Preservation

Iron Mountain And CyArk Join Forces To Digitally Preserve Fort York For Generations To Come. Press Release. IT Business Net. October 04, 2017.
      Fort York is a historical site laden with rich Canadian history. To ensure future generations can continue to learn about and experience the site, it is being preserved in an online virtual library, along with other world heritage sites.  The technology uses 3D laser scanning, photogrammetry, and traditional survey techniques to create an online, 3D library of the world's cultural heritage sites before they are lost to natural disasters, destroyed by human aggression, or ravaged by the passage of time.  In addition, the project will also use virtual reality technology to "transform Fort York into a living legacy". These efforts are part of how they ensure cultural heritage sites will be available for future generations to experience, while making them uniquely accessible today.

Saturday, October 07, 2017

Church Preserves Precious Records of African Nation

Church Preserves Precious Records of African Nation. Newsroom. The Church of Jesus Christ of Latter-day Saints. 28 September 2017.  [YouTube]
     In Freetown, Sierra Leone, paper records dating back to the early 1800s are disintegrating at an alarming rate due to poor storage conditions, heat, humidity and frequent handling. The staff often pool money just to keep the lights on one or two days a week and were making a “frantic effort” to preserve copies of records by hand. “I had my heart broken because of the conditions of how these records are kept and the way that the people are working here, giving the best of themselves to preserve what they can for people and families of Sierra Leone.”  Despite valiant efforts by dedicated caretakers, rampant deterioration of the tattered records threatened to obliterate the very history of the nation. This changed with a plea from the government of Sierra Leone to the LDS president, asking for help in preserving the at-risk records.

The Church approved a project to image the dilapidated birth and death records and make them available online. FamilySearch has began the process of digitizing records in Freetown and in towns and remote villages across the country, and works with interfaith leaders such as The Catholic Church which has opened its record vaults to be part of the preservation project. “Any kind of record at all is crucially important because it becomes a database for future generations.” The preservation project will preserve some 4 million records.  FamilySearch is engaged in similar digitizing projects in countries all over Africa.

Friday, October 06, 2017

Exploring Digital Preservation, Digital Curation, and Digital Collections in Mexico

Exploring Digital Preservation, Digital Curation, and Digital Collections in Mexico. Natalie Baur. bloggERS! February 22, 2017.    
     This is a post about international perspectives on digital preservation and relating to the state of digital preservation initiatives and digital information access in Mexico. Interviews were conducted with organizations working on building, managing, sharing and preserving their digital collections. The findings indicate:
  • The focus on digital collection building and preservation in business and government tends toward records management approaches. 
    • "While the guidelines and processes for paper records are robust, many institutions are only beginning to implement and use electronic records management platforms."
    • Long-term digital preservation of permanent records designated is an ongoing challenge.
  • In academic, archives, and cultural heritage institutions, digital collection work is focused on digitization, storage and access of digitized assets, as well as issues related to long-term, sustainable maintenance of digital collections.
  • While digital preservation issues are still in the early stage of development. In Mexico and Latin America, this has mostly been done through participation in the InterPARES project. Several Mexican academic and government institutions have taken the lead on digital preservation issues, and a more cohesive, intentional organization progress in the near future.
"It is now more crucial than ever for librarians, archivists, developers, administrators, and program leaders to look outside of the United States for collaborations and opportunities to learn with and from colleagues abroad." We need to share resources, perspectives, diversity, and world wide improvements.

Thursday, October 05, 2017

Exploring Metadata Interoperability in the United States and United Kingdom

Exploring Metadata Interoperability in the United States and United Kingdom. Charlotte Kostelic. bloggERS! March 28, 2017.    
     A post about international perspectives on digital preservation. It looks at a comparative analysis of descriptive metadata for collections and specifically understanding how metadata can aid in providing access to digitized collections and inter-operable access for the collections. One goal of this analysis was to find a common data model for the various collections.
  • The standards used by the partner institutions include: 
    • Encoded Archival Description (EAD) with DACS for archival collections in the United States; 
    • ISAD(G) for archival collections in the United Kingdom; 
    • MARC for bibliographic, map, serial, and print collections; and 
    • Dublin Core employed for certain digital collections records. 
    • There are additional library and museum standards that need to be analyzed further.
  • Key access points include: subject headings; dates; languages; and place, personal, and corporate names.
  • The level of description between collections varies based on whether or not the materials are from archival collections or library collections.
There is a need for inter-operability between collections that use different data models, especially in an institution that intends to make all collections accessible in a single viewer.

Wednesday, October 04, 2017

Electronic Records Management Guidelines: Long-Term Preservation

Electronic Records Management Guidelines: Long-Term Preservation. March 2012, Version 5. Minnesota Historical Society. [Guidelines - Contents]
     Over the course of time, organizations generate many records. Some are of short term duration while others are to be kept permanently. “Tools such as migration, conversion, metadata, and eXtensible Markup Language (XML) will help you not only preserve your records, but also realize their full value.” Records need to be preserved, since “the greatest possible access to certain government information and data is essential to allow citizens to participate fully in a democratic system of government.”   

Some key concepts presented in the section on Long-Term Preservation:
  1. Needs Assessment. Understanding the value of the records and the information they contain will help guide decisions, determine their retention requirements, the access and use of the records, as well as preservation options. 
  2. Physical Storage Options. Record access requirements will help determine the type of storage to use, specifically
    1. Online storage. Immediately available on the network
    2. Near-line storage.  Records are stored in automated optical disk or tapes libraries attached to a network.
    3. Offline storage.  Records are stored on removable media that must be retrieved manually.
  3. File Format Options. For long-term file preservation, non-proprietary formats are preferred, but they also have limitations.
  4. Digital Preservation Techniques. There are several approaches to ensure that electronic records remain useful over time.
    1. Emulation. Using emulator programs to simulate the behavior, of original programs.
    2. Encapsulation. Combining the object to be preserved with all of the necessary details of how to interpret it within a wrapper or package.
    3. Migration. This is the more common approach, which is the process of  moving files to new media or computer systems to maintain their use. 
  5. Preservation Planning.  “A preservation plan should address an institution’s overall preservation goals and provide a framework that defines the methods used to reach those goals.  At a minimum, the plan should define the collections covered by the plan, list the requirements of the records, practices and standards that are being followed, documentation of policies and procedures related to preservation activities, and staff responsibility for each preservation action.” This plan needs constant updating and cost/benefits must be addressed. Policies should be developed to put the plan into practice.

Tuesday, October 03, 2017

Consortial Certification Processes: the Goportis Digital Archive: a Case Study

Consortial Certification Processes: the Goportis Digital Archive—a Case Study. Franziska Schwab, Yvonne Tunnat, and Dr. Thomas Gerdes. bloggERS! February 7, 2017.
     Another blog post on the international perspectives on digital preservation and about digital preservation in Germany and repository certification . The certification of their digital archive is part of the quality management and all workflows are evaluated. The certification process shows that the long-term availability of the data is ensured, and the digital archive is trustworthy. They have completed the certification processes for the Data Seal of Approval (DSA) which took over six months. They are working on the application for the nestor Seal which is much more complex and requires more detailed information.

Monday, October 02, 2017

Developing a Citizen Archive

Developing a Citizen Archive.  Anssi Jääskeläinen, Miia Kosonen, and Liisa Uosukainen. bloggERS! February 1, 2017.
     A blog post on the international perspectives on digital preservation and about digital preservation in Finland. They believe there is a need for a digital preservation service to give citizens the right to decide what to do with their personal information. They believe there is a problem of just using cloud storage and are creating the Citizen Archive to solve this problem. "Cloud storage services especially have surged in popularity in recent years, but these services are not OAIS-compliant, have no support for metadata schema such as METS and PREMIS, and make no guarantee that the data or user-generated metadata uploaded will remain safe or searchable."

 This is a personal archive which provides more than just information storage and retrieval. "It represents other important values, such as legacy building, protecting against loss of important personal data, and constructing personal identity". Another important aspect of modern family heritage is in providing digital interaction between family members; they have developed a workflow to convert Outlook emails to PDF/A files. They are also trying to address the social, technical, and legal challenges of long-term storage and maintenance.

Friday, September 22, 2017

Electronic Records Task Force Phase 2 Final Report

Electronic Records Task Force Phase 2 Final Report. John Butler, et al. University of Minnesota. August 23, 2017. [PDF, 68 pp.]
     The University of Minnesota Libraries sponsored an Electronic Records Task Force to monitor established workflows and to develop new workflows, policies, procedures and mechanisms for processing and providing access to electronic records. They are focused on the development of processing activities, best practices and guidelines. Creating finding aids, which are published online through ArchivesSpace, are the first step in providing access to electronic records. The long-term preservation of electronic records is a concern and this effort continues to be a work-in-progress. To keep up with the influx of electronic records, the Electronic Records Task Force provides the following recommendations:
  1. Staffing: Hire a permanent full time employee to work exclusively with electronic records
  2. Long-term Management: Create an Electronic Records Management Group to address ongoing electronic records needs
  3. Preservation: Review current workflows and long-term management requirements to address immediate and long-term solutions for file backup, recovery, and preservation according to policies and standards
  4. Security: Conduct a thorough review of security requirements
  5. Equipment: Establish initial and ongoing financial support for hardware, software and collections
  6. Access to Materials: Explore options for providing access to electronic records, including both access and preservation of these materials.
Project Tasks and Deliverables
  1. Develop Workflows for Processing Ingested Collections
  2. Define Processing Levels (minimal, intermediate, full)
  3. Develop Access Methods that Address End-user Needs, Copyright, Data Privacy and other Information Security Requirements
  4. Monitor Ingest Workflows and adjust as necessary
Additional notes:
  • "In the long-term, a full-time dedicated staff person is the most responsible approach to working effectively and efficiently, to achieve quality work, and to maintain our leadership role in the field of electronic records management. This is arguably the only way to address the ingest and processing activities that assist with long-term access to and preservation of electronic materials. Without a dedicated person who has an in-depth understanding of evolving workflows and protocols and who can provide a consistent approach with curatorial staff, any headway in addressing the records being collected will be made slowly."
  • The goal of processing unique electronic archival material is to make it available to end users, whether they be skilled researchers or a high school student working on a project.
  • Given divergent requirements, a singular asset management, backup, and preservation solution may not be a feasible goal in either the near or long term. However, efforts can be made to establish a limited number of processes to manage the vast majority of preservation use cases.

Friday, September 15, 2017

Preservation with PDF/A

Preservation with PDF/A (2nd Edition). Betsy A Fanning. DPC Technology Watch Report 17-01. July 2017. [PDF 34pp.]  [Link updated]
     This report is an updated edition of the original Technology Watch Report 08-02, Preserving the Data Explosion: Using PDF (Fanning,2008). It looks at PDF/Archive as digital document file format for long-term preservation. The PDF/A versions of the PDF format have been developed as a family of open ISO Standards to address preservation of PDF files by removing features that pose preservation risks. It is important for preservation purposes to know how closely a file conforms to the  requirements defined in the standard. There are preservation risks that may exist in the standard PDF file format:
  • any file type can be embedded;
  • the primary document can be conformant as a static document, but the embedded files may not be static;
  • embedded files may be infected by computer viruses;
  • embedded files may have extended metadata requirements, may introduce unexpected dependencies or be subject to format obsolescence;
  • embedded files may complicate matters relating to information security, data protection or the management of intellectual property rights.
By restricting some risk features and thus reducing preservation risks, the PDF/A format seeks to maximize:
  • device independence;
  • self-containment;
  • self-documentation.
Some reasons why an organization might use PDF/A to preserve their digital documents, include:
  • its standardized format for storing digital documents for long periods of time;
  • it allows for digitally signed documents using the very latest digital signature software;
  • it reliably displays special characters for mathematics and languages since all are embedded within the file;
  • it displays correctly on any device as the author intended, including the reading order;
  • platform independence;
  • provision of fully searchable documents through Optical Character Recognition.
History and Features of PDF and PDF/A. The Standard was drafted in multiple in order to make it easier to implement the Standard. "Unfortunately, the committee’s philosophy of multiple parts resulted in confusion in the market place, making it more difficult for users to select the optimum file format." Users  may need to do a file format assessment based on their requirements that can help them decide which PDF/A Standard to implement.

Metadata helps effectively manage a file throughout its life cycle, as well assist in document discovery searches. "Establishing a long-term digital document preservation system requires careful consideration of the metadata that will be needed to locate and render documents years from now." Collecting metadata for the PDF/A documents in optional in the standard, except for the identifier, which is generated when the PDF/A file is created. Preservation metadata should:
  • be appropriate to the materials;
  • support interoperability;
  • use standardized controlled vocabulary;
  • include clear statements on the conditions and terms of use;
  • be authoritative and verifiable;
  • support the long-term management of the document.
Just because a file purports to be a PDF/A does not necessarily mean that it is. Format validation of a file can increase confidence a viewer will be able to render the file correctly.  A number of PDF/A validators are available.The development work on the PDF Standards is a continuing effort. There are additional preservation challenges in the format that are in the process of being addressed.

The report lists some recommendations, which are directed at groups that use the standard. They include:
  • For those evaluating PDF/A as a digital preservation solution:
    • Before adopting PDF/A as a preservation solution it is "essential to understand the organizational requirements and how PDF/A will support" the organization needs.
    • PDF/A is not a preservation solution on its own a part of the wider preservation strategy that must be consistent with other components of the preservation infrastructure, such as backups, integrity checks and documentation.
    • Different versions of PDF/A have different purposes, with different capabilities as well as different preservation risks. These should be understood and decisions should be documented and explained.
    • Different vendors offer different tools to manage PDF/A that should be compared against your requirements..
  • For organizations collecting and preserving digital data:
  • While it may not be possible to control or restrict how documents are produced, it may be useful to give document creators guidance on what is desired.
  • Embed PDF/A validation tools into preservation workflows and record the results to help manage the digital preservation risks associated with PDF/A files received.

Wednesday, September 13, 2017

Self-preservation: The Gibraltar National Archives uses cloud to safeguard its history

Self-preservation: The Gibraltar National Archives uses cloud to safeguard its history. Caroline Donnelly. ComputerWeekly. 13 September 2017.
     Many enterprises are familiar with the concept of retaining corporate data as part of their regulatory and compliance obligations. But some fail to understand that the data must be kept accessible. "While regulatory compliance is the key reason why many enterprises embark on this process in the corporate world, for the Gibraltar National Archives (GNA), digital preservation is an essential part of ensuring the annals of its cultural heritage and democratic history are safeguarded forever." After a long process of digitizing historical content, they realized that digitising content is not the same as preserving it. "The risk was we could have spent all this time and money doing digitisation only to lose [this information] a few years down the line because it is not preserved correctly.” Digital preservation is about:
  • actively managing the file formats
  • ensuring they remain readable in future
  • being proactive and managing the content
Just as it is important to be able to prove the provenance of physical records, the fixity of the digital documents needs to be maintained.  “People often ask me when our digital preservation project will be finished. I tell them never, because every day we are collecting records. Every day we are archiving unique material from newspapers to government records all for generations to come.”

Saturday, August 19, 2017

IBM and Sony cram up to 330 terabytes into tiny tape cartridge

IBM and Sony cram up to 330 terabytes into tiny tape cartridge. Sebastion Anthony. Ars Technica UK. August 2, 2017.
     IBM and Sony have developed a new magnetic tape system capable of storing 201 gigabits of data per square inch, or approximately 330 terabytes in a single palm-sized cartridge. To achieve this density, Sony developed a new type of tape that has a higher density of magnetic recording sites, and IBM Research developed new heads and signal processing technology to process the data from the "nanometre-long patches of magnetism". The new cartridges and tape drives, "when eventually commercialised, will be significantly more expensive because of the tape's complex manufacturing process."

Friday, August 18, 2017

Evaluating Your DPN Metadata Approach

Evaluating Your DPN Metadata Approach.  DPN Preservation Metadata Standards Working Group. July 27, 2017. [PDF, 6 pp.]
     This brief guide can help determine a clear metadata approach to recovering data "in the far future among unpredictable circumstances".  The document can help users create a sound approach to preserving your institution’s data and make decisions that fit with their own institutional needs.

The first section is:
What information is needed to understand and contextualize an object? It examines both descriptive and structural metadata.

Descriptive Metadata: for the purpose of identification and discovery of an object. Dublin
Core, MODS and VRAcore are common standards used for descriptive metadata.  

Structural Metadata: describes relationships between objects, such as pages in a book. The METS Structural Map can express  hierarchical relationships or parent/child relationships. The PREMIS "relationship" element can express version relationships.

The document also looks at how to:
  • understand and contextualize a collection; 
  • connect/relate objects to a collection; 
  • connect/relate versions to each other; 
  • connect metadata records to associated objects and collections;
  • ensuring the authenticity of an object;
  • ensuring the essential characteristics of the original are maintained in a data migration

Thursday, August 17, 2017

DPN: Metadata Considerations for Deposits

Metadata Considerations for Deposits. DPN. August 2017.
     The Digital Preservation Network working groups have provided an overview of the types of metadata to consider while preparing deposits for DPN. Several areas are addressed:
  1. DPN-specific metadata, especially DPN-specific metadata, DPN’s BagIt specification, Tag Directories and Bag Structure.
  2. DuraCloud-specific metadata, while they do not restrict metadata they "indicate that local policies should be used to define metadata approaches".  Each snapshot contains four DuraCloud-created files: checksums (md5, sha265), a content properties file, and a collection-snapshot file  
  3. Core descriptive metadata records. The DPN Preservation Metadata Standards Working Group examined minimal metadata records from a variety of member institutions to find common metadata schemas. This resulted in  a “core record,” or the "minimum level of information needed in order to understand digital assets at a later date," shown in a clear chart.
  4. Significant properties of content. "In order for digital files to be usable and accessible in the long-term, it is important to recognize the importance of significant properties and to ensure that the properties of your digital materials are being documented in some form." They list content types, with examples of common significant properties. 

Tuesday, August 08, 2017

Universal Electronic Records Management Requirements

Universal Electronic Records Management Requirements. Courtney Anderson. National Archives Records Express. August 4, 2017.
     The National Archives has released the Universal Electronic Records Management Requirements as part of the Federal Electronic Records Modernization Initiative (FERMI). Universal ERM Requirements identify high level business needs for managing electronic records. The program requirements are derived from existing NARA regulations, policy, and guidance and are a starting point for agencies to use when developing system requirements. "Records management staff should work with acquisitions and IT personnel to tailor any final system requirements". The document contains an abstract, a glossary, and lists of lifecycle requirements and transfer format requirements.
There are six sections based on the lifecycle of electronic records management:

1.    Capture
2.    Maintenance and Use
3.    Disposal
4.    Transfer
5.    Metadata
6.    Reporting

The requirements are either “program” requirements, relating to the design and implementation of policies and procedures, or “system” requirements, providing technical guidance for creating or acquiring ERM tools, which also indicate “Must Have” or “Should Have”. NARA will be supporting these requirements going forward and will be updating them to stay current with changes in technology, regulations and guidance products.

Saturday, August 05, 2017

Elsevier Acquires bepress

Elsevier Acquires bepress. Roger C. Schonfeld.  Society for Scholarly Publishing; The Scholarly Kitchen. Aug 2, 2017.
     Elsevier announces its acquisition of bepress. In a move entirely consistent with its strategy to pivot beyond content licensing to preprints, analytics, workflow, and decision-support, Elsevier is probably the foremost single player in the institutional repository area. There is some concern this acquisition will allow them to co-opt open access. The bepress product, Digital Commons, has more than 500 participating institutions, predominantly US colleges and universities.

bepress Joins Elsevier, with Exciting Potential for Growth. Press release. bepress. Aug 2, 2017.
bepress has joined Elsevier, the largest content provider in the world. The management is "confident that this is the right choice for bepress and for our community. Both parties are committed to sustaining the elements that make bepress bepress, and supporting your open access initiatives."

Thursday, August 03, 2017

Library Preservation Workflows: Importing, Exporting, and Managing Content

Library Preservation Workflows: Importing, Exporting, and Managing Content. Chris Erickson. June 12, 2017. [PDF slides]
     This is my presentation at the Eight Annual Rosetta's Advisory Group meeting held in June at the wonderful University of Sheffield. This is my favorite conference because of the attendees, the topics discussed, the interaction with the Ex Libris employees who attend, and the many things I learn about digital preservation and Rosetta, in the Advisory meeting and in the accompanying Rosetta Users Group. I hate to see this conference end.

The short presentation is a view of some of the ongoing changes and refinements we have made to our digital preservation workflow in the past year. We have worked to streamline our processes, both because of the increased volume of content we ingest into Rosetta, and also the desire to minimize the file movement and copying during the processing. In addition, we have used our preservation repository to recover documents in our access systems that became unavailable.

During the year we have updated our digital preservation policies to help determine our preservation selection workflow. They include:
The changes have helped with a smoother transition from selection and acquisition, processing and SIP creation and submission to Rosetta, and the preservation disposition.

Wednesday, August 02, 2017

Digital Preservation Workflow Curriculum

Digital Preservation Workflow Curriculum. Mary Molinaro. DPN, AVPreserve. August 2, 2017.
     DPN and AVPreserve have developed a "digital preservation workflow curriculum to share with DPN members and others in the digital preservation community". This workshop curriculum, released with a Creative Commons license, will provide participants with skills and knowledge to implement and manage a digital preservation program within their organization. They ask that the terms of the CC-BY-SA license be observed.

The workshop modules show the requirements of a digital preservation ecosystem from the viewpoints of governance / program management, as well as asset management. This is not an introduction to digital preservation or the OAIS model; instead it looks at the 'why' and 'how' questions of "making digital preservation an underlying, operational function of an organization". The curriculum, which is available here in a zip file, consists of:

Friday, July 21, 2017

ePADD 4.0 Released

ePADD 4.0 Final. July 21, 2017.
    This is the latest release of ePADD, a software tool "developed by Stanford University's Special Collections & University Archives that supports archival processes around the appraisal, ingest, processing, discovery, and delivery of email archives."

The software is comprised of four modules:
  1. Appraisal: Allows users to gather and review email archives 
  2. Processing: Tools to arrange and describe email archives.
  3. Discovery: Tools to share a view of email archives with users through web discovery 
  4. Delivery: Enables repositories to provide access within a reading room environment.
System Requirements:
  • OS: Windows 7 SP1 / 10, Mac OS X 10.10 / 10.11 
  • Memory: 8 GB RAM (4 GB RAM allocated to the application by default) 
  • Browser: Chrome 50/51, Firefox 47/48 
  • Windows installations: Java Runtime Environment 64-bit, 8u101 or later required
ePADD Installation and User Guide
ePADD Github website

Saturday, July 15, 2017

Email preservation: How hard can it be?

Email preservation: How hard can it be? Edith Halvarsson. Digital Preservation at Oxford and Cambridge. 7 July, 2017.
      The post summarises highlights of the Digital Preservation Coalition’s briefing on email preservation. What is email? It is "an object, several things and a verb”, a heavily linked and complex object, like the web. "Retention decisions must be made, not only about text content but also about email attachments and external web links. In addition, supporting features (such as instant messaging and calendars) are increasingly integrated into email services and potential candidates for capture."
Email is also a cultural and social practice; capturing relationships and structures of communication is an additional layer to preserve. 

What is being done, or can be done?  Migration is the most common approach to email preservation. EML and Mbox, which is a family of formats, are the most common formats migrated to. They have  different approaches to storing content. Others choose to unpack content which provides a way to display emails and normalise content within them. The emulation approach provides access to content within the original operating environment. Also, ePADD, an open source tool, provides functions for processing and appraisal of Mbox files, but ha other features

There are still questions and issues still to explore, particularly regarding web links. "Email archives may be more valuable to historians as they acquire critical mass".  Some thing that institutions can do are:
  • Participate with the  Email Preservation Task Force
  • Share your workflows to the Email Preservation Task Force and the community
  • Run trial migrations between different email formats such as PST, Mbox and EML and blog about your finding
  • Support open source tools such as ePADD and make them sustainable! 

Friday, July 14, 2017

Six Priority Digital Preservation Demands

Six Priority Digital Preservation Demands. Somaya Langley. Digital Preservation at Oxford and Cambridge. 13 July, 2017.
     Post discusses the gap between what activities need to be done as part of a digital stewardship end-to-end workflow and the maturity level of digital preservation systems. It presents a list of "my six top ‘digital preservation demands’ (aka user requirements)":
  • Integration with  other systems: A digital preservation ‘system’ is only one piece of a much larger puzzle. In the ‘digital ecosystem’  end-to-end digital stewardship workflows are of primary importance. Metadata and/or files should flow from one system to another.  
  • Standards-based:  Libraries rely on standards. "If we don’t use (or fully implement) existing standards, this means we risk mangling data, context or meaning; potentially losing or not capturing parts of the data; or just wasting a whole lot of time".
  • Error Handling: With more work and few people, we "have to be smart about how we work. This requires prioritisation." The preservation workflows need smarter systems to aid the processes, especially understanding and resolving errors from the many third-party tools. 
  • Reporting: The types of reports needed include: 
    • High-level reporting – annual reports, monthly reports, reports to managers, projections, costings etc.)
    • Collection and preservation management reporting 
    • Reporting for preservation planning purposes, based on preservation plans
  • Provenance: Support for identifying where a file has come from. This is often handled by metadata and documenting changes as Provenance Notes. The essential metadata (administrative, preservation, structural, technical) needs to be captured and retained.
  • Managing Access Rights:  We must ensure we can provide access to the content to support both the content and users in a variety of ways, particularly the new ways they want to use the content. 
"It’s imperative to keep in mind the whole purpose of preserving digital materials is to be able to access them...." Addressing these six concerns may not be easy, but we need to "make iterative improvements, one step at a time."