Born Digital: Guidance for Donors, Dealers, and Archival Repositories

Born Digital: Guidance for Donors, Dealers, and Archival Repositories. Gabriela Redwine, et al. Council on Library and Information Resources. October 2013. [PDF]
     "Until recently, digital media and files have been included in archival acquisitions largely as an afterthought." People may not have understood how to deal with digital materials, or staff may not be prepared to manage digital acquisitions. The object is to offer guidance to rare book and manuscript dealers, donors, repository staff, and other custodians to help ensure that digital materials are handled, documented appropriately, and arrive at repositories in good condition, and each section provides recommendations for donors, dealers, and repository staff..

The sections of the report cover:

• Initial Collection Review
• Privacy and Intellectual Property
• Key Stages in Acquiring Digital Materials
• Post-Acquisition Review by the Repository
• Appendices, which include: 
• Potential Staffing Activities for the Repository
• Preparing for the Unexpected: Recommendations
• Checklist of Recommendations for Donors and Dealers, and Repositories

Some thoughts and quotes from the report:

• it is vital to convince all parties to be mindful of how they handle, document, ship, and receive digital media and files.
• Early communication also helps repository staff take preliminary steps to ensure the archival and file integrity, as well as the usability of digital materials over time.
• A repository’s assessment criteria may include technical characteristics, nature of the relationship between born-digital and paper materials within a collection, information about context and content, possible transfer options, and particular preservation challenges.
• Understand if there is a possibility that the digital records include the intellectual property of people besides the creator or donor of the materials.
• Clarify in writing what digital materials will be transferred by a donor to a repository
  (e.g., hard drives, disks, e-mail archives, websites)
• It is strongly recommended that donors and dealers seek the
  guidance of archival repositories before any transfer takes place.
• To avoid changing the content, formatting, and metadata associated with the files, repositories
  must establish clear protocols for the staff’s handling of these materials.

The good practices in this report can help reduce archival problems with digital materials. "Early
archival intervention in records and information management will help shape the impact on archives of user and donor idiosyncrasies around file management and data backup."

Digital Preservation Matters.

Risk management guide for the secure disposal of electronic records

Secure destruction of electronic records. Archives New Zealand. 2 October 2015.
     Blog post on the secure and complete destruction of electronic records plus all copies and backups. Destruction of paper records are mostly straightforward. However it is not so easy to confidently delete electronic records. The processes to destroy digital records should be secure, irreversible, planned, documented and verifiable.  The article has examples of risks of not destroying records,  as well as resources on how to implement the destruction records. In addition there is a new guide on the benefits of disposal and the risks of not disposing of records: Risk management guide for disposal of records.

[Disposal and destruction of digital records may not seem like it has anything to do with digital preservation, but it is an important part of records management. More than just that, it can be a needed part of the submission and ingest processes made multiple copies of sensitive content have been created before or while adding the content.  Or if you have been given media to add and then must dispose of the media afterwards. -cle]

Digital Preservation Matters.

New Case Study: Digital Preservation Strategies for a Small Private College

New Case Study: Digital Preservation Strategies for a Small Private College. Meg Miner. Society of American Archivists. May 19, 2015.
     Well established “best practices” in digital preservation do little to address day-to-day realities in repositories that do not have the money or staff for digital preservation activities. The question then is how to provide good stewardship for digital content before a complete preservation system is in place?

They assumed their institutional repository (IR) would work as a preservation platform. After attending a NEDCC workshop about digital preservation program attributes, it became apparent that IWU’s repository, DigitalCommons, hosted by bepress,  did not meet the requirements for a full DP system. It lacked included bit-level analysis on ingest and during storage, file format normalization, and a means for detecting and replacing corrupt files.

The article is an in-depth explanation of actions taken by IWU’s archivist prior to the POWRR Project and the workflows established as a result of it. The Archives does not currently have a full scale preservation solution, but insights from the POWRR helped establish digital records’ documentation practices and storage strategies.

In 2009, files were copied to a 5-disk Redundant Array of Independent Disks (RAID) drive that was monitored by library IT. One disk failed in 2014 and the entire RAID was replaced with one of reduced capacity and relocated from the library. Several project members were looking for a quick, simple solution, but from the workshop they realized no tool will replace making decisions about which historical records hold significance to our institutions. These values and individual behaviors are what the cultural heritage community truly needs to spend time on.

Anne Kenney and Nancy McGovern on digital preservation: “A fully implemented and viable preservation program addresses organizational issues, technological concerns, and funding questions, balancing them like a three-legged stool.”

A lack of answers for everything does not mean "being free to stand by and do nothing. If support for a full preservation program is unlikely, there are less resource intensive ways to provide good stewardship for digital records". The lessons learned in the Digital POWRR workshop and confirmed by practices now in place show that "slight modifications to familiar accession workflows will create an audit trail and prepare digital objects for bit-level preservation storage". It is important to start documenting our decisions today so that future repository managers who will inherit the outcomes of our work, will be able to carry these objects into the next generation of preservation products.

Digital Preservation Matters.

Measure for Measure: Tracking Effort in Born Digital Processing

Measure for Measure: Tracking Effort in Born Digital  Processing. Erika Farr. CurateGear 2015. January 7, 2015.
     Presentation at CurateGear on measuring and tracking efforts. The percent of effort for each task was helpful. Some notes from the slides:

FY2014 Metrics Gathering

• Three Digital Archives staff members processing born digital collections
• Established categories for measuring effort
• Obtain Data, Virus Check
• Ingest
• Appraisal and Developing Processing Plan
• Collection Processing (Arrangement and Description, Privacy and Restrictions)
• Prepare for Access 
• Used Redbooth for business collaboration tool

Processing Metrics in FY2014

• Processed born-digital material in 5 collections
• Processed over 2 GB worth of data, over 4100 files
• 388 hours of effort
• On average the processing completed over 5MB per hour, 18 files per hour.

Digital Archives Effort by Task

• 4% Obtain Data
• 1% Ingest
• 19% Appraisal and Processing Plan
• 71% Collection processing
• 5% Prepare for Access

Metrics are useful for:

• Creating and modifying workflow and deciding the tools to use
• Business planning for acquiring collections, staffing, budgeting, and prioritizing 
• Determining the total cost of collections

Digital Preservation Matters.

Arkivum: Long-term bit-level preservation of large repository content

Arkivum: Long-term bit-level preservation of large repository content.  Nik Stanbridge. Arkivum. DSpace User Group. 16 June 2015. [PDF slides]
     Based on the Principles of ‘Active Archiving’, which is replication, escrow, and integrity
checking. Trying to preserve content for longer than 25 years. Principles based on diversity, intervention, with different technologies and locations.

• Adding media, a continual process
• Monthly checks and maintenance updates
• Annual data retrieval and integrity checks
• 3-5 year obsolescence of servers, operating systems and software.
• Tape format migration

Integration with DSpace. It has ISO 27001 validated processes and procedures and is designed for bit level preservation for large volumes of data.

Similar posts:

• Release of Archivematica 1.4 and Storage Service 0.7
• Chronopolis and DuraCloud: Doing integration right
• New Sources and Storage Options For Rosetta
• Finnish Digital Preservation Service for Cultural Heritage

Digital Preservation Matters.

A Digital Preservation Environment Maturity Matrix for NSLA Libraries

A Digital Preservation Environment Maturity Matrix for NSLA Libraries. Sarah Slade, David Pearson, Libor Coufal. iPres Proceeduings. October 2014.
The National and State Libraries of Australasia (NSLA) established a Digital Preservation Group to understand the state of digital preservation in the various libraries and to determine the core requirements for managing the preservation of digital collections. They listed and described the functional components of an ideal digital preservation environment and created a matrix of the current stage of development against each component for each NSLA library. Related projects by others include the National Digital Stewardship Alliance and BenchmarkDP.

NSLA Digital Preservation Environment Maturity Matrix

• Underlying Assumptions
• actively collecting digital material
• committed to preserving its digital materials for the long term.
• staff (or vendor) dedicated to the project
• intends to comply with OAIS

• Functional Component

An ideal digital preservation environment should contain a mix of policies, processes and resources (including staff and technologies). The OAIS model calls for organizations to:

• Negotiate for and accept information from information producers.
• Obtain sufficient control of the information for long-term preservation.
• Determine the designated user community.
• Ensure the information is independently understandable to the designated community without the need of special resources.
• Follow documented preservation policies and procedures.
• Make the information available to the designated community.

Instead of just listing the functions, they created a set of questions about the OAIS functionality in order to help responders describe their organization's level of digital preservation maturity, and included the OAIS functions:

• Pre-ingest Activities
• Ingest
• Archival Storage
• Data Management
• Administration
• Digital Preservation Planning
• Access

• Maturity Model

The Group modified the Capability Maturity Model, using 5 levels:

1. Initial. Processes are usually ad hoc. Achievement depends on the competence of the people in the organization and not on the use of proven processes. Products and services usually exceed budget and schedule
2.  Repeatable. Basic digital preservation processes are established.  Digital preservation achievements are repeatable, though not all activities.
3. Defined. Digital preservation activities are performed and managed according to documented plans. Processes for digital preservation are established and improved over time. 
4. Managed. Management can effectively control the digital preservation effort, using precise measurements.  Quantitative quality goal for digital preservation processes
5. Optimising. Organisation focuses on continually improving process performance. The effects of deployed digital preservation process improvements are measured and evaluated against the quantitative process-improvement objectives.   

"NSLA has identified digital preservation as an area of priority. The importance of this area to NSLA libraries is reflected in the creation of the Digital Preservation Group and its support of the Group’s work to date. The results from the Digital Preservation Environment Maturity Matrix reveal that NSLA libraries are on the right path but have some way to go before digital preservation processes are mature, sustainable and fit for purpose. Collaboration on policies, products and infrastructure will continue to address these needs."

Digital Preservation Matters.

Finnish Digital Preservation Service for Cultural Heritage

Finnish Digital Preservation Service for Cultural Heritage. Mikko Tiainen. PASIG Presentation. March 12, 2015. [PDF]
Preservation aspects and focus of the Digital Preservation Service:

• Semantic preservation:
• Content knowledge and semantics
• Descriptive metadata
• Logical preservation
• Preservation planning
• Administrative & technical metadata
• File formats
• Preservation actions
•  Bit-level preservation
• Materials & replication management
• Storage device
• Storage media

They estimate the life cycle of the various components as:

• Hardware
• Hard disk storage: 5 years
• Tape drives & media types: 5 years
• Tape libraries: 10 years
• Software
• Commercial support at  least for 5 years
• Open source maintained and developed until replaced 

Development of a high quality digital preservation system is a continuous process. 

• Bit-level preservation
• Preservation planning and development of preservation actions
• Preserving the intelligibility
• Distributed locations

This requires support services, maintenance of specifications, and ongoing management.

 Digital Preservation Matters.

Format Migrations at Harvard Library: An NDSR Project Update

Format Migrations at Harvard Library: An NDSR Project Update. Joey Heinen. The Signal. April 17, 2015.
Digital material is just as susceptible to obsolescence as analog formats. There are different strategies that can be implemented, and they are developing a format migration framework for migration projects at Harvard. The viability of this framework will be tested by migrating three obsolete formats within the Digital Repository Service: Kodak PhotoCD, SMIL playlists and RealAudio. A first step is determine the stakeholders and responsible parties, since a digital preservation project cannot begin without knowing the stakeholders.

A diagram of the Migration Workflow shows each step of the process from gathering documentation for initial analysis to ingest of the migrated content into the repository. A Migration Pathway diagram shows how content will be transformed by a migration. They hope that analyzing the technical and infrastructural challenges of each format and putting this into a template that can be adapted will help the digital preservation field.

 Digital Preservation Matters.

Tech talk in the archives: how can we redefine our processes & priorities in the digital age?

Tech talk in the archives: how can we redefine our processes and priorities  in  the  digital  age?  Erin  O’Meara. PASIG Presentation. March 12, 2015. [PDF]
Tech talk for archives usually revolves around workflow, data clean up and translation, infrastructure needs, and digital content pathways. Traditional archives have a physical focus, work is long term and ongoing, and priorities are established based on perceived use. For digital archives, the focus is on the digital objects, which tends to be more of a project based approach that needs dependency analysis before work is done. Prioritization is based on a grid of preservation needs, use and access, and an impact on the larger repository.

Get to know your complete digital holdings (servers as well as boxes of disks); the infrastructure, your capabilities for acquiring, processing and preserving digital holdings; staff and their training needs; and the gaps in all of this. Your processes need to change from "as-is" to "to-be".

• Gain momentum and resources "building the ship while flying"
• Geta business analysis and outsider perspective
• Manage processes and know when to change
• Continue building, and allow the archive to grow and mature
• Allow time to review and reflect
• Clarify and integrate roles between archive and tech staff

In determining your directions:

• State the  larger goal then document  it
• Break it down into executable chunks
• Engage and educate stakeholders and leadership
• Talk to colleagues outside your workplace
• Curiosity and tinkering  encouraged
• Give your team a break and recognition

 Digital Preservation Matters.

Digitization Challenges – A Discussion in Progress

Digitization Challenges – A Discussion in Progress. Merrilee Proffitt. Blog. OCLC Research. March 23, 2015.
There are challenges faced by libraries digitizing collections, such as dealing with born digital materials, storage and preservation, web harvesting, and others. Their recent discussion looked at these topics:

• Metadata: Item-level description vs collection descriptions. The challenge is digitizing archival collections at the item or page level when the descriptions are at a collection level. How can we engage scholars to help with the description if the resources are outside the library?
• Process management / workflow / shift from projects to programs. There are challenges to establish workflows to meet different needs. Some are transitioning from projects to programs.
• Selection – prioritizing users over curators and funders. "Many institutions are still operating under a model whereby curators or subject librarians feed the selection pool" even though surveys indicate selection should move towards directly serving the needs of the users.
• Audio/Visual materials. Making these available is a concern because differing levels of interest, high costs, reformatting capacities, and need for accompanying transcriptions. 
• Access: are we putting things where scholars can find them. Are collections discoverable from Google or the institution? What are the users' experiences in using the collection?

Digital Preservation Matters.

Significance 2.0: a guide to assessing the significance of collection

Significance 2.0: a guide to assessing the significance of collections. Roslyn Russell, Kylie Winkworth. Collections Council of Australia Ltd. 2009.
This guide is for defining an adaptable method for determining significance across all collections in Australia. The intention is that it will improve collection decision-making in areas such as, preservation, access, and funding support. Regarding significance:

• We cannot keep everything forever. It is vital we make the best use of our scarce resources for collecting, conserving, documenting and digitising our collection materials.
• Significance is not an absolute state; it can change over time.  
• Collection custodians have a responsibility to consult communities and respect other views in constructing societal memory and identity. 
• It is vital to understand, respect and document the context of collection materials that shape collection materials.

Significance’ refers to the values and meanings that items and collections have for people and communities. Significance helps unlock the potential of collections, creating opportunities for
communities to understand, access and enjoy collections. Artistic, scientific and social or
spiritual values are the criteria or key values that help to express how and why an item or collection is significant. Part of the criteria are: provenance, rarity or representativeness, condition or completeness, and interpretive capacity. Significance assessment involves five main steps:

1. analysing an item or collection
2. researching its history, provenance and context
3. comparison with similar items
4. understanding its values by reference to the criteria
5. summarising its meanings and values in a statement of significance

 A statement of significance is a concise summary of the values, meaning and importance of an item
or collection. It is an argument about how and why an item or collection is of value. This should be reviewed as circumstances change.  Significance assessment is

• a process to help with good management of items and collections; 
• it is a collaborative process and consultation is essential.
• it will substantiate justify assessments objectively rather than subjectively

The process is:

1. Collate information about the history and development of the collection
2. Research the history, scope and themes of the collection
3. Consult knowledgeable people
4. Explore the context of the collection
5. Analyse and describe the condition of the collection
6. Compare the collection with similar collections
7. Identify related places and collections
8. Assess significance against the criteria
9. Write a statement of significance
10. List recommendations and actions

Significance assessment is only the first part of the significance process. Once an item or collection has been assessed as significant, there will be a range of actions to better manage the collections.
 

Digital Preservation Matters.

Preserving progress for future generations

Preserving progress for future generations. Rebecca Pool. Research Information. February/March 2015.
Digital preservation remains one of the most critical challenges facing scholarly communities today. From e-journals and e-books to emails, blogs and more, electronic content is proliferating and organizations worldwide are trying to preserve information before the electronic information is lost. Some of the organizations include: Portico (which preserves content on behalf of participating publishers; the number of open access journals it includes is rising, ); CLOCKSS (still grappling with the cost models of providing preservation service).

There is a rising demand for the preservation of dynamic content. No one is able to "capture dynamic content and [preserve] a day-to-day, or even, minute-to-minute feed of this content." There are only snapshots. CLOCKSS is developing the ‘how to’ process to preserve these ‘snapshots’ across multiple locations, validating each against the other, and is also exploring the best pricing structures to preserve such content.

Other organizations include LOCKSS, The Digital Preservation Network, HathiTrust, Preservica, Archivematica, and Rosetta, whose recent clients are the State Library of New South Wales and the State Library of Queensland.

The digital preservation development is clearly gaining momentum, growing in both size and complexity. "Clearly progress is being made and you can measure that by the maturity of solutions on offer." But for most organizations, the urgency of digital preservation has yet to hit home.

"Trying to sell the idea of digital preservation on the basis of return on investment has been very hard. By its nature, it’s a long-term activity and you’re really hedging your bets against future risks. I think we are still in the very early days of genuinely understanding the value of digital assets... and transferring this understanding over to financial assets doesn’t yet work very well." The European consortium 4C (Collaboration to Clarify the Costs of Curation) has been investigating this problem. Their road map helps organisations appraise digital assets, adopt a strategy to grow preservation assets and develop costing processes. In addition they have developed a model for curation costs. The only way to understand the costs of preservation is though sharing, through openness and collaboration.

Digital Preservation Matters.

Digital Audio Preservation at MIT: an NDSR Project Update.

Digital Audio Preservation at MIT: an NDSR Project Update. Susan Manus, Tricia Patterson. Library of Congress; The Signal. January 16, 2015.
Report of the residency position, in which Tricia is primarily tasked with: completing a gap analysis of the digital preservation workflows currently in place for audio streaming and preservation, and developing lower-level diagrammatic and narrative workflows. [Workflow images are in the article.] Workflow documentation is receiving increased acknowledgement and appreciation in the preservation environment. The reasons:

• tested, repeatable road map allows processing of larger projects with efficiency and security
• detailed workflows show redundancies and deficiencies in processes across departments
• workflow documents clarify roles and accountability within the chain of custody.

The benefits include getting a better idea of what digitization project documentation is generated and that the documentation needs to be preserved as well. It has also helped identify steps that would benefit from automation.The process started with itemizing 50-60 delivery requirements, including relevant TRAC requirements (PDF), covering display and interface, search and discovery, accessibility, ingest and export, metadata, content management, permissions, documentation and other considerations. From there requirements were prioritized on a scale from “might be nice” to “must-have.” The next step is to measure options against our prioritized requirements to determine the needs of the Libraries now. An important part is to provide meaningful access to the audio treasures in the library.

Digital Preservation Matters.

Aligning Customer Needs: Business Process Management (BPM) and Successful Change Management in the L. Tom Perry Special Collections

Aligning Customer Needs: Business Process Management (BPM) and Successful Change Management in the L. Tom Perry Special Collections. Joseph Gordon Daines III. Library Leadership & Management. November 2014. PDF
A lot of archival processing happens before an archival or manuscript collection can be made available for research use by patrons. It is central to the archival endeavor. This article looks at the role of business process management (BPM) in automating many of the workflows used to manage the manuscript and archival collections

BPM is a field of management focused on aligning organizations with the needs and wants of their customer bases. The Special Collections department identified its customer bases as its curatorial staff and its patrons. Enabling the curatorial staff to more efficiently prepare manuscript collections for research use would also enable better customer service. Several different BPM techniques were used to gain an understanding of the curatorial needs of  the department as it automated the  workflows. This enabled the department to successfully simplify and streamline its workflows during the course of automating them. The end result has been more efficient processing of archival collections and better service for our patrons.

A review of the requirements showed a need for two types of functionality:

1. task management, and 
2. archival content management.

Business process:

• Systematic management, measurement and improvement of all company processes through cross-functional teamwork and employee empowerment.
• Standardize activities and processes in order to improve organizational efficiency
• Business process: “a series of interrelated activities, crossing functional boundaries, with specific inputs and outputs.”
• The tools that will be examined are process mapping, process modeling, statements of work, and use cases. Processes are modeled using at least one of the following charts: general process charts, process flow diagrams, process activity charts, or flowcharts.
• Flowcharts are useful in identifying decision points and parallel activities in a process.gain an understanding of the sequence of activities in the process
• Statements of Work: A specific statement regarding the requirements needed in a service contract. The statement of work should include all aspects of job requirements, performance and assessment.
• Use cases also help identify the actors involved in various activities and what they want from those activities. For the purposes of use cases, actors are defined as “anything that interfaces with your system—for example, people, other software, hardware devices, data stores, or networks. Each actor defines a particular role.” Use cases typically include two components—a diagram featuring the actor(s) and how they interact with the system and a flow of events statement. The flow of events statement is “a series of declarative statements listing the steps of a use case from the actor’s point of view.”
• ProcessMaker provides a SOW template that aided the project in automating the workflow comprising the department’s implementation of the archival business process.
• The use of BPM tools and techniques in the Perry Special Collections provided the department with a methodology to examine and improve the workflow used to provide access to archival materials.
• Business processes enable leaders to make informed decisions that can improve library’s abilities to deliver their services. 
• BPM tools are not difficult to use and provide a wide range of benefits. Library leaders should use BPM tools to lead successful change initiatives.

Digital Preservation Matters.