Weekly readings - 17 March 2006

Excuse Me... Some Digital Preservation Fallacies?  Chris Rusbridge.  Ariadne.  February 2006.

http://www.ariadne.ac.uk/issue46/rusbridge/

The article looks at a number of issues with digital preservation that the author feels are fallacies.  They are:

   1. Digital preservation is very expensive [because]

   2. File formats become obsolete very rapidly [which means that]

   3. Interventions must occur frequently, ensuring that continuing costs remain high.

   4. Digital preservation repositories should have very long timescale aspirations,

   5. 'Internet-age' expectations require the preserved object must be easily and instantly accessible in a useable format, and

   6. the preserved object must be faithful to the original in all respects.

 

All preservation, including paper and book preservation, is expensive.  Digital preservation as a whole compared to paper and book needs may actually be less.  While consumer formats may go out of fashion, very rarely are any formats that are completely obsolete.  Recovery of information from old files can be incomplete.  Mass access to the internet has stabilized the formats.  Part of the key to this is to share the information.  This may be more of a problem with extended time frames.  “Investment in digital preservation is important for cultural, scientific, government and commercial bodies. Investments are justified by balancing cost against risk; they are about taking bets on the future. The priorities in those bets should be: first, to make sure that important digital objects are retained with integrity, second to ensure that there is adequate metadata to know what these objects are, and how they must be accessed, and only third to undertake digital preservation interventions.”

 

It may not be necessary to look at digital preservation in hundreds or thousands of years.  What institutions have this timescale?  It may be more useful to look at digital preservation as a series of events or a relay.  Make your decisions on the timescale that you can see and that you have the funding for.  Preserve your objects to the best of your ability and hand them intact on to your successor.  The right approach may be to keep the original bits and then produce access copies as you can.  The high cost of accessing the original may be best given to the user who asks for them. 

 

A restatement of the original issues would be:

   1. Digital preservation is comparatively inexpensive, compared to preservation in the print world,

   2. File formats become obsolete rather more slowly than we thought

   3. Interventions can occur rather infrequently, keeping costs down.

   4. Digital preservation repositories should adjust their timescale to meet their funding and business case, but should be prepared for their succession,

   5. "Internet-age" expectations cannot be met by most digital repositories; and,

   6. Only access versions of the preserved object need be easily and instantly accessible, although the original file and good preservation metadata should be available

 

The lack of money is the biggest obstacle to effective preservation.  Poor decisions will reduce the amount of material that can be preserved. The right choice may be “fewer and better” or “cheaper and more”. 

 

 

---

 

Future-Proofing Web Sites.  Maureen Pennock.  Ariadne.  February 2006.

http://www.ariadne.ac.uk/issue46/dcc-fpw-rpt/

The DCC workshop goal was to provide insight about ensuring ongoing access to web sites over time. This is not just a matter of archiving, but also about how to design and manage a web site so that it is suitable for long-term preservation with minimum intervention.  In one presentation, the key to this is the three R's -Reduce, Replicate and Redirect. Reduce the items to make them easier to preserve, replicate them in multiple formats, and redirect links to the new locations.  It is more ‘future-improving’ rather than ‘future-proofing’.  There need to be selection criteria and guidelines to collect and preserve web sites as part of an organization’s wider preservation strategy.  Standards should be applied preferably at the point of creation rather than a later time.  Persistent identifiers and important, but we should be looking at 15 – 20 years, not longer.  Metadata should  document the technical dependencies and tools; this is more useful than just descriptive metadata. The method of selecting web sites must also be documented. 

 

Some record management principles require the documents to be saved but not necessarily the web site itself.  An organization can therefore determine what needs to be saved, but it may not have to be the entire site.  There should be a clear delineation of tasks and responsibilities.  The National Library of Australia introduced PANDAS 3, a software tool for managing the process of gathering, archiving, and publishing web site resources.  Authenticity is a key issue for web sites.  Preservation management  must include three key aspects: passive preservation; active preservation; and managing multiple manifestations.  Permission should be obtained before archiving web sites.  The main issues were:

 

·         think about the records perspective;

·         reduce, replicate and redirect;

·         protect your domain;

·         be archive-friendly;

·         carry out 'not-bad practice';

·         experiment, and;

·         identify unhelpful practice.

 

---

 

Decision Tree for Selection of Digital Materials for Long-term Retention.  Deborah Woodyard-Robinson.  Digital Preservation Coalition.  March 8, 2006. 

http://www.dpconline.org/graphics/handbook/dec-tree.html

This is an updated version of a decision tree, which is a tool to construct or test such a policy an organization.  The questions and choices in the tree will assist with the decision to accept or reject long-term preservation responsibility.  An effective policy must also be:

·         Endorsed by senior management

·         Actively circulated throughout the organization

·         Reviewed regularly

·         Accompanied by an appropriate resource commitment

        PDF of the Decision Tree (47KB)

 

Weekly readings - 10 March 2006

University researchers develop new digital rights technology.  Jaikumar Vijayan.  Computerworld.  March 10, 2006.

http://www.computerworld.com/securitytopics/security/story/0,10801,109449,00.html?source=NLT_AM&nid=109449

Researchers at the University of Maryland have developed a new digital rights management technology to better protect multimedia content from unauthorized copying and distribution.  The technology embeds a unique ID or fingerprint on individual copies of multimedia content. It is designed to allow owners to trace the content, even if it is pieced together from multiple copies.  This can be applied to images, video, audio and other types of documents. 

 

 

 

Editors' Interview with Victoria Reich, Director, LOCKSS Program.  RLG DigiNews.  February 15, 2006. 

http://www.rlg.org/en/page.php?Page_ID=20894#article1

The LOCKSS (Lots of Copies Keep Stuff Safe) Program offers libraries a cost effective and easy way to build digital collections of Web-based content. Digital information is extremely fragile and preservation must start from the moment it is put into circulation.  Components of LOCKSS include:

·   Replicate the content in independent repositories.

·   Audit the digital content is fragile. If files are continuously compared and damage automatically repaired, off-line back up can be eliminated.

·   A hands-off approach. Minimal processing is needed 

·   Open source software is critical.

·   Allow no single points of failure. Strive for diversity in administration, funding, and technology.

·   Have extremely cost-effective processes.

The LOCKSS system can preserve content in any format available over the Web as long as it has a stable URL structure, and changes at a moderate pace. The single greatest threat to materials being preserved over the long term is money.  Preservation must be accomplished at marginal expense to avoid the threat of scarce economic times.   The CLOCKSS (Controlled Lots of Copies Keep Stuff Safe) Initiative is designed to test the feasibility of a large, community-managed dark archive.   Members are working towards a production system. 

 

“The fundamental goal of a digital preservation system is that the content stored in the system remains accessible to future readers for a time much longer than the lifetime of any individual component of the system.”  Digital preservation is concerned with long timeframes.  All information system components are unreliable in the long run. A fundamental design principle of a digital preservation system is that it must tolerate component failures.  LOCKSS has planned for format migration, obsolescence, scalability, and it’s own possible demise.  While future access can never be proven, it can work to increase the odds that the content will be available in the future.

 

 

 

Hitting the ground running: building New Zealand’s first publicly available institutional repository.  Nigel Stanger, Graham McGregor.  University of Otago.  09 March 2006.

http://eprints.otago.ac.nz/274/01/dp2006-07.pdf  

Institutional repositories are becoming more important.  This low cost and fully functional repository was started and went live in 10 days.  It has received a very large number of hits.  The repository was built with ePrints in three phases: technical implementation, content collection and administration.  They decided to restrict the pilot to voluntary contributions in PDF format their business school.  This effort was publicized to the department heads to get early acceptance, and they quickly received materials, mostly departmental working or discussion papers which already had permission to publish online.  Items with uncertain copyright status were restricted until the status was confirmed. They found the SHERPA website valuable for copyright information.  They decided to follow Dublin Core Metadata to make it compatible with other projects.  They were able to establish the repository quickly because it was a proof of concept and not a large scale project that involved many disciplines or other people. Policy and procedural issues which needed institutional decisions were noted rather than addressed.  They used a minimalist approach to the effort, especially with gathering content. The site which is on the internet has had over 18,000 downloads.  The repository, which contains about 220 items, shows what can be done by a dedicated team.

 

 

 

OpenDoc Prescription a Bitter Pill for Microsoft in Massachusetts. Richard Entlich.  RLG DigiNews.  February 15, 2006. 

http://www.rlg.org/en/page.php?Page_ID=20894#article3

An in depth discussion on Massachusetts requiring the OpenDocument format for new office applications.  The document specified that  “executive branch agencies would be required to migrate office document software to applications able ‘to save office documents by default in the OpenDocument format’ by January 1, 2007 and that ‘any acquisition of new office applications must support the OpenDocument format natively.’ The only other acceptable format mentioned in the document was PDF.”  This informative article also provides a chronology of the events.  It is unknown where this effort will lead, but it has already had an impact as it shows others the need for open formats.  Digital preservation is a process of risk management.  To date, much of the preservation efforts have been reactive, but they need to become more proactive.  These actions may have unforeseen consequences, but without taking a chance we may never know what degree of change is possible.

 

 

 

Weekly readings - 03 March 2006

NEDCC Survey and Colloquium Explore Digitization and Digital Preservation Policies and Practices.  Tom Clareson.  RLG DigiNews.  Feb 15, 2006.

http://www.rlg.org/en/page.php?Page_ID=20894

The Northeast Document Conservation Center conducted an online survey to develop a way to assess institutions’ digital preservation readiness. While digitization efforts are increasing, there are a lack of policies to deal with these materials once they are created.  While a majority of institutions had policies dealing with goals, collection development, and emergency preparedness, few of them address the digital holdings specifically.  IT staff are key to the success of digitization projects.  A majority of institutions are involved in digital imaging projects, and over half provide online searching to the public.

·        29% of respondents had a policy on the creation of digital resources

·        63% said that 5% of their budget or less was devoted to any type of preservation activities.

·        9% had no funds whatsoever allocated for preservation activities.

·        31.1% did not have an IT department

·        92% had created digital assets from physical source materials, mostly from flat paper or images, also books, and AV

·        39% said the majority of the items they consider to belong to digital collections are unique, single-copy works

·        66.9% provide access to digital collections through an institutional website

·        83% had created descriptive metadata for the digital assets to help find and use of digital collections

·        50% at least also created technical and administrative metadata

·        25% do not assign any portion of their budget to create digital collections

·        42% do not have budget lines for acquiring digital collections

·        60%  do not have a specific person assigned responsibility/primary activity for digital preservation

·        84% supported staff development and professional education/training for digital preservation, but it does not seem to translate into policy development

·        30% of collections are not adequately protected by a backup strategy.

·        52.8% said they do not insure their digital holdings, while 36.5% did not know

·        29% of the institutions responding to the survey have policy, planning, or procedure documents on the creation of digital resources

 

Responses to the means of digital preservation included regular data backup, migration, and refreshing the data; maintaining legacy equipment and disks, outsourcing to an externally-managed repository, and emulation. Storage media include network hard drives (78%), or removable magnetic media (65%). Digital collections are most often stored in-house in systems managed by the institution.  A preservation study concluded that “small and medium-sized institutions will need the assistance of experts to assess the preservation status and needs of their expanding digital collections.”

 

 

 

ILM isn’t even clear to it’s OWN user community.  Larry Medina.  Computerworld.  February 17 2006.

http://cwflyris.computerworld.com/t/312105/1109184/9784/0/

Information Lifecycle Management (ILM) is still causing major confusion for many people because there is a question as to what it is. It is being promoted as something new.  Some vendors who push their version of ILM may use scare tactics and mis-information to say that the best  way to accomplish this is to save everything.  While it may be easier right now, it is not the cheapest or best way.  The best way is to evaluate exactly what you need to keep and how long you need to retain it.  Use this knowledge to create a policy and train your organization to manage the information effectively.  It also requires a classification system, and the ability to assign record series to the information.  If the information has a long retention period, you will also need to choose appropriate formats and media.  This method will minimize the information being kept, which will mean shorter indexing times,  faster searches, smaller repositories, quicker backups, lower risks during e-discovery, and a uniform method of managing the information.  Until the practices and responsibilities are defined, “throwing a ‘canned solution’ at an improperly analyzed problem is foolish.”

 

 

 

Microsoft hit with fresh charges over Office, future products.  Simon Taylor.  Computerworld.  February 2, 2006. 

http://www.computerworld.com/governmenttopics/government/legalissues/story/0,10801,108888,00.html?source=NLT_PM&nid=108888

Some of Microsoft’s rivals have charged that it is shutting out competitors.  One of the issues raised is Microsoft's refusal to disclose interoperability information for its Office suite. They are refusing to provide data such as the file formats for .doc, .xls, and .ppt documents, which prevents rival application suites from achieving full compatibility.  This has crucial implications Linux systems. 

 

 

 

Open Document Format (ODF)... let the discussion begin!  Larry Medina.  Computerworld.   March 3, 2006. 

http://www.computerworld.com/blogs/node/1914?NLT_DM_B

In commenting on the Open Document Format discussion, the author cites the Wall Street Journal article: “The data belongs to the people, not to the software vendor that created the file format.”  The most important observation is that a standard controlled by a single company is not a standard. 

 

 

 

A Microsoft Document Fight Brews; IBM, Sun Join New Group Promoting Common Format For Government Records.  Wall Street Journal.  March 3, 2006.

http://online.wsj.com/article/SB114135713113288409.html?mod=opinion_main_com

As alliance of software vendors plan to promote use of the OpenDocument Format, a set of software technologies for storing and creating documents.  The Open Document Alliance includes IBM, Sun, Oracle, ALA, and 30 others.  This heightens the debate over whether governments should adopt software that supports the OpenDocument Format.   Microsoft doesn’t support the format.  Backers argue that the format  is more trustworthy for storing documents because it isn't owned by a single company.  Microsoft is using a new format called OpenXML in its Office 12 software, which is also supported by Apple and Intel.  This is a result of the actions in Massachusetts last year when the state's information-technology division decided to standardize its programs on the OpenDocument Format.

 

 

 

Weekly readings - 24 February 2006

The AIHT at Stanford University: Automated Preservation Assessment of Heterogeneous Digital Collections. Richard Anderson, et al. D-Lib Magazine. December 2005.

The Stanford Digital Repository is a set of services to provide to eventually preserve any digital content deemed institutionally significant or valuable. The early process focus on preserving a subset of the potential content. The group focused on tiered levels of service for digital preservation which they hoped would provide the flexibility to adapt to changing technologies to manage and preserve digital materials and their descriptions. The technical metadata seemed easier to create automatically but the contextual would be difficult. JHOVE was used for technical analysis and metadata creation. Sustainability and quality & functionality are the primary forces making any format "preferred”. Their preferred formats by type are:

Plain text: ASCII, UTF-8;
Marked-up text: XML 1.0;
Image: TIFF 5.0 and above (uncompressed);
Page-Viewer: PDF (any version);
Audio: WAVE (linear pulse code modulation); and
the Video format was yet to be determined.

“An automated assessment process is clearly the only efficient means to collect technical information about large numbers of files.” The knowledge will help to measure risk, to negotiate with depositors, and to make decisions about the long term preservation. In general, their experience showed that practical tests are valuable for understanding the needs and making informed decisions.

---

UKWAC: Building the UK's First Public Web Archive. Steve Bailey, Dave Thompson. D-Lib Magazine. December 2005.

We depend on the internet in many ways, but we pay little attention to the long-term preservation of websites. Invaluable scholarly, cultural and scientific resources are being lost to future generations. Six leading UK institutions are working on a project to test selective archiving of UK websites. The group has chosen a modified version of the PANDAS software, developed by the National Library of Australia. Their goals include:

· To work collaboratively in the achievement of a common searchable archive of selected web sites investigating solutions to issues such as, selection, rights management and digital preservation

· To evaluate the development of the collaborative infrastructure for web archiving with regards to assessing the permanence and long-term feasibility of such a collaborative enterprise

Archiving web sites follows the basic archival principles of Selection, Acquisition, Description and Access. Individual Partners in the group select web sites to be archived, but there are some additional steps to this process. The partners will check that someone else has not already selected a particular web site for archiving. When a new site is selected, basic metadata is entered into the central database; a group member then becomes responsible for that site's life cycle management. They seek explicit written permission to archive sites from site owners before archiving the site. There are difficulties and challenges in archiving web sites. “Web archiving is not an exact science.” In spite of the difficulties, this has been an important project for digital preservation. It has shown that selective web archiving can be done through using a consortium, it has highlighted the fragility of web-based materials while offering a workable solution. http://www.webarchive.org.uk/

---

Mind the gap: assessing digital preservation needs in the UK. Martin Waller, Robert Sharpe. Digital Preservation Coalition, 2006.

This 'state of the nation' report today shows that less than 20% of UK organizations surveyed have a strategy in place to deal with the risk of loss or degradation to their digital resources – even though there is a very high level of awareness of the risks and potential economic penalties. The survey shows that digital data loss is commonplace and seen by some as inevitable.

· Over 70% of respondents said data had been lost in their organization

· 87% recognized that key material could be lost and

· 60% said that their organization would lose financially

· 52% of the organizations said there was management commitment to digital preservation

· Only 18% had a digital preservation strategy in place

Some high profile instances include: The decision that Morgan Stanley must pay over $1 billion for failure to preserve and hand over some documents required by the courts; the 1975 Viking Lander mission data tapes have deteriorated despite careful storage, and scientists are unable to decode the formats used. The principal risks to digital material are:

· the deterioration of the storage medium;

· obsolescence of hardware, software or storage format;

· failure to save crucial document format information, such as preserving tables of numbers without preserving an explanation of their meaning.

The report identifies 18 core needs and recommendations to address those needs. The needs include:

· Increase awareness of digital preservation issues especially among data creators;

· Take stock of digital materials (55% of those surveyed did not know what digital material they had;

· Fund digital preservation aspects of projects from the beginning ;

· Increase funding for digital archives

“Gone are the days when archives were dusty places that could be forgotten until they were needed. The digital revolution means all of us – organisations and individuals – must regularly review and update resources to ensure they remain accessible. Updating need not be expensive, but the report is a wake-up call to each one of us to ensure proper and continuing attention to our digital records.”

It is important to create long-term pro-active preservation plans, and allocate adequate budget and resource to implementing practical solutions. “Organisations that create large volumes of digital information need to recognise the benefits of retaining long-term information in digital form so that these can be balanced against the costs of active preservation.”

Weekly readings - 17 February 2006

The Archive Ingest and Handling Test: The Johns Hopkins University Report. Tim DiLauro, et al. D-Lib Magazine. December 2005.

Johns Hopkins University (JHU) performed the Archive Ingest and Handling Test with two repository applications, DSpace and Fedora. Their model consisted of two classes of objects (data-group and data-item), which consisted of identifier, name, metadata, and group and item ids. They used METS as a wrapper for the metadata for a digital object. They generated the SIP packet and then ingested it into the repositories. The bulk ingestion was extremely slow; DSpace and Fedora had constraints on the process. Errors, memory limits, and server issues caused crashes. The size of the collection was also a factor. In the second phase, JHU exported the data to Stanford and imported the data from Harvard. Each of the participants chose a different approach for their dissemination packet, but there would have been an advantage to having common elements in the process. In the format transformation phase, their goal was to create a flexible mechanism to allow systematic migration of specified content. They chose to migrate JPEGs to TIFFs; they added metadata about this conversion to the TIFF itself, in addition to the item metadata. The problems with DSpace and Fedora were reported.

Lessons learned included:
· The log files produced during ingest, export, and migration are important and should be structured so they can be used later.
· After a bulk operation there should be an easy way to rollback the changes.
· Memory consumption was a big issue.
· When processing the objects, it was necessary to access objects one at a time and write any intermediate results to disk.
· The export was complicated by storing the METS in each DataObject; instead it should have been assembled from the content and then reassembled during export.
· The configuration information for the repositories should have been stored in one config file and referred to symbolically.
· Be prepared for tools not working perfectly.
· The metadata provided was inconsistent.
· Format registries will form the basis for future automated processes for ingestion and management / migration of content already in repositories.

---

Archive Ingest and Handling Test: The Old Dominion University Approach. Michael L. Nelson, et al. D-Lib Magazine. December 2005.

Old Dominion University (ODU) was the only non-library to participate in the test. The focus was on
· self-archiving objects
· archive models & granularity
· archive export & import

They used the MPEG-21 Digital Item Declaration Language (DIDL) complex object format They do not have an institutional repository so the process is more of a pre-ingest phase. The metadata descriptors follow the general Dublin Core structure. The file processing workflow is represented in the article. The imported items are given a new identifier based on the MD5 of the file name. There is an additional MD5 checksum generated on the contents of the object for verification needs. Each object was processed with JHOVE to provide technical metadata, and also checked in the Format Registry Demonstrator. Format conversion is handled as a batch process for archived items and the work flow is outlined. Through this process, they regarded the repository as “a preservation threat the data must survive.”

---

Plextor Ships Industry's First 18x DVD±R Burner. Press Release. Computer Technology Review. February 13, 2006.

Plextor has announced a DVD±R/RW CD-R/RW drive aimed at users who require reliability, high performance, and premium recording features. This drive has a recording speed of 18X DVD±R on certified 16X DVD±R media. It also supports lower speeds.

---

Plumbing and storing e-archives: an industry blooms. Brian Bergstein. January 31, 2006.

With the increase of digital communications, memos, presentations and other bits of data are increasingly finding their way into "electronic discovery" centers. "The big risk for companies is too much data that there's really no business need for, being kept in ways that if they had to go looking for it, would be uneconomic." "In litigation today, if e-discovery is done wrong, it can have huge implications.” Other times evidence comes not from what's in a file, but from its "metadata" -- the automatically applied labels that explain such things as when a file was made, reviewed, changed or transferred.

Weekly readings - 10 February 2006

AIHT: Conceptual Issues from Practical Tests. Clay Shirky. D-Lib Magazine. December 2005.

The Archive Ingest and Handling Test (AIHT) is a project of the National Digital Information Infrastructure and Preservation Program (NDIIPP), sponsored by the Library of Congress. The idea is that by giving a complete digital archive to a variety of participants, we can better understand which aspects of digital preservation are general and which are institution-specific. It was also to:

· test and assess the feasibility of transferring digital archives from one institution to another,
· document useful practices,
· discover which parts of the handling of digital material can be automated, and
· identify areas that require further research or development.

The George Mason University's collection of 9/11 materials was selected. Some of the lessons learned are:

· In ingest, the identifiers must be independently verified, such as determining whether files labeled .jpg are really in JPEG format.
· The desirability of a digital object will be more closely related to the value of its contents than to the quality of its metadata.
· Donors are likely to force preserving institutions to choose between accepting imperfectly described data or receiving no data at all.
· Even a small percentage of exceptions in operations on a large archive can create a large problem.
· Maintaining digital materials for a short term is relatively simple, but more difficult for the long term
· It is possible for a preserved bit stream become unusable because the complex the system changes over time: hardware, software, OS, etc.
· Multiple preservation strategies provide the best hedge against unforeseen systemic failure.
· Using a variety of strategies to preserve data may be a better strategy than putting all efforts toward one single preservation system.
· There is a need for continual comparative testing of preservation tools and technologies.

---

Harvard's Perspective on the Archive Ingest and Handling Test. Stephen Abrams, et al. D-Lib Magazine. December 2005.

Harvard has operated a preservation repository for over 5 years which contains over 3 million objects, and takes over 12 TB of space. The repository is intended for “highly ‘curated’ digital assets; that is, those that are owned and submitted by known users, created according to well-known workflows and meeting well-known technical specifications, and in a small set of approved formats.” They intend to eliminate the restrictions and make it an institutional repository. Some of the issues they will face in the future include:

· Automated extraction of technical metadata from digital objects
· Automated generation of Submission Information Packages (SIPs)
· Systematic preservation migrations
· Post-migration quality assurance testing
· Metadata models for capturing provenance information

The input method used the JHOVE program to validate file types and to generate some of the technical metadata. They also chose to migrate image files to JPEG 2000. The data model did not provide a way to capture provenance metadata, but they decided they would look at PREMIS for this purpose. The format migration process was fully automated once the appropriate specifications were developed. In general, the test showed that in spite of scaling problems, “digital content can be transferred without loss between institutions utilizing radically different preservation architectures and technologies.” The limiting factor in transferring large amounts of data appears to be the number of objects, rather than their individual or total size.

---

Research books its place in the library of the future. IST Results. 1 Feb 2006.

Digital preservation is one of the three major research areas of the European Digital Library. Audiovisual material is particularly vulnerable to being lost due mostly to technological obsolescence. In the PrestoSpace project, they found 60 different video formats, which increases the problems. This project, with Media Matters, has created a method of transferring the obsolete media into digital data. Other parts of the project include a database listing all the known characteristics of types and years of video tapes, and an algorithm for the restoration of video and audio materials.

---

Technology victim: Western Union sends its last telegram. Todd R. Weiss. Computerworld. February 03, 2006.

Western Union has delivered its last telegram messages. This means of communication began over 155 years ago but has been replaced by other means of communication. Over 200 million telegrams were delivered in 1929, and only 20,000 were delivered last year.

---


Interview as learning tool… Michael Yunkin. digitize everything. February 3rd, 2006.

Some comments from an interview about digitization and preservation that are worth reading. Excerpts include: Digitization is not preservation, except possibly the source material is fragile, and using a digital surrogate can help avoid overuse. "The increased access that comes with digitization IS added value."

Weekly readings - 3 February 2006

Libraries fear digital lockdown. Ian Youngs. BBC News. 3 February 2006.

The British Library has warned that digital rights management (DRM) controls may block some legitimate uses of digital books and journals and that they may not be accessible in the future when technology evolves. The estimate that by 2020, 90% of newly published work will be available digitally. If these works are locked, it could cause problems in the future. "It is probable that no key would still exist to unlock the DRMs. For libraries this is serious.” Libraries, as “custodians of human memory”, would need to “keep digital works in perpetuity and may need to be able to transfer them to other formats in order to preserve them and make the content fully accessible and usable once out of copyright.”


---


Film preservation. Stephen Galloway. Hollywood Reporter. Dec. 20, 2005.

The Academy of Motion Picture Arts and Sciences' Pickford Center for Motion Picture Study is housed in a 118,000-square-foot former television studio that has become a state-of-the-art hub for film preservation and restoration. The activity has increased dramatically recently because of government and private funding that has provided the means to keep film from becoming extinct. "There is more money and more public awareness about preservation in general." The center, which houses over 100,000 films, works on about 200-300 films per year and have improved or re-mastered 2,000 titles, 1,000 of them have had full restorations. There is also a commercial market for maintaining these films, so some of the studios are beginning to invest in preservation instead of turning the material over to archives or the public. Creating an inventory and finding the materials is the most challenging part of the process. Restoring the sound can be more challenging than restoring the picture on some movies. A major issue in preserving movies would be the creation of a standard to store the movies digitally instead of on celluloid. The Digital Motion Picture Archival Project, comprised of studio and archival experts, met in June.

---

Lossless JP2 Meta-Standard for Video Archiving. Websites. January 18, 2006.

These two websites contain information on Jpeg2000 and video archiving. The first is a new forum for discussing the documents and issues. The second site contains some documents that explain the MPEG-A Process, and overview of the approach for creating a new Meta-Standard for Long-Term Digital Video Preservation, and the first draft of the standard.
Follow-on activity - Towards an MPEG-A meta-standard for lossless JPEG-2000-based video compression.

---


Trends in archiving. Raju Buddharaju. Computerworld (Singapore). December 2005.

There are a number of issues to be considered when archiving web sites. The technological issues include obsolescent formats and access mechanisms, legal and organizational issues such as copyright issues and managing intellectual property. Economic issues include long term viability and the cost of web archiving. There are some archiving models that look at these factors; the models look at closed and open archives, and also look at collection schemes such as selective archives, thematic, or deposit and domain archives.

There are four major archiving tasks, such as, a search and selection process, an ingest process, a storage facility, and access mechanisms. The tools used for each of these tasks vary for each model. For example, for closed web archives, the archived web sites are pre-selected, and with a closed group, getting copyright issues are simpler. Open archives need policy guidelines on which web sites to archive, how much to gather, what the copyright processes are, and how often to get the web site information. Ingest mechanisms could include converting original formats into archival formats to make storage and access easier.

There are a number of web archiving initiatives in place, such as Pandora in Australia, the Austrian Registration of On-Line Archive Project, Preservation and Access to Czech Resources on the Internet, Net Archive Project in Denmark, EVA project in Finland, BnF’s project for web archiving in France, Warp project in Japan, Paradigma project in Norway, Kulturarw3 project in Sweden, the Domain UK project, and Minerva in the US. Some focus on archiving the country’s specific domains. But in general at least 40 per cent of the links in these web sites refer to web sites outside of the country’s specific domains. As an alternative, international or consortium-based web archiving approaches are used. These projects archive websites be either making an archival copy, or by taking a snapshot of the web site periodically.

Weekly readings - 27 January 2006

It May Look Authentic; Here's How to Tell It Isn't. Nicholas Wade. The New York Times. January 24, 2006.
http://www.nytimes.com/2006/01/24/science/24frau.html?_r=1&pagewanted=all&oref=login

In the digital age, photo-manipulation has proved particularly troublesome for science and others. In recent events, photos have been manipulated to misrepresent scientific tests. The Journal of Cell Biology editors recognized the likelihood that images were being improperly manipulated when the journal required all illustrations to be submitted in digital form. The guidelines state that nothing should be done to any part of an illustration that did not affect all other parts equally. Basically that it is all right to adjust the brightness or color balance of the whole photo, but not to obscure, move or introduce an element. Others are concerned about the authenticity of digital images. The editors are developing methods of detecting certain patterns of image manipulation.





EMC adds Google Desktop search to Documentum. Lucas Mearian. Computerworld. January 17, 2006.
http://www.computerworld.com/hardwaretopics/storage/story/0,10801,107811,00.html?source=NLT_SU&nid=107811

EMC announced that Google Desktop has been added to the Documentum 5.3 Enterprise Content Integration (ECI) Services. The search feature can access unstructured content stored on desktops, including documents, Web pages, XML, and audio and video files. Other Google applications for web and network have already been supported.





Data-Storage Essentials. Washington Post. From PC World. December 24, 2005.
http://www.washingtonpost.com/wp-dyn/content/article/2005/12/23/AR2005122300180.html

For consumers the ideal storage is ultra cheap DVDs and CDs, which can store a lot of files permanently. For moving files around from one place to another, flash memory or an external hard drive are very useful. An external hard drive is good for backing up your data.




DoD First to Transfer Digital Records to National Archives. Steven Donald Smith. January 23, 2006.
http://www.defenselink.mil/news/Jan2006/20060123_3987.html

The Defense Department became the first federal agency to transfer digital records to the National Archives and Records Administration using the new established Electronic Records Archive "pre-accession" guidelines. These guidelines st

Weekly readings - 20 January 2006

Building the UK's First Public Web Archive. Steve Bailey, Dave Thompson. D-Lib Magazine. January 2006. http://www.dlib.org/dlib/january06/thompson/01thompson.html

In spite of our dependence on the internet, very little attention has been paid to the long-term preservation of websites. To remedy this, six UK institutions are developing a test project to archive UK websites. The article discusses their experience and the lessons they are learning from the project. The process follows the basic archival principles of Selection, Acquisition, Description and Access. The project has had some success, but there are also some very real difficulties that they have encountered. They have had difficulty in finding a suitable program. Some systems have no descriptive standards or no subject control. Some of the sites change so rapidly it is difficult to work with them. Technical skills are required to repair ‘broken’ sites. The project has allowed the technical IT staff to interact with the archival staff, giving them opportunities for greater collaboration. Selective archiving ensures maximum functionality and accurate and faithful reproduction of the sites. “The result is a high quality archive, whose content is selected along clear and agreed lines, that is robust and authoritative.” The site is at http://www.webarchive.org.uk/


Nikon to focus on digital cameras. BBC News. 12 January 2006. http://news.bbc.co.uk/1/hi/business/4605418.stm Nikon has said that it will stop making most of its film cameras and focus on digital cameras instead. This will allow it to meet the competitive needs in the digital market. Nikon said that the film market has continued to decrease while the digital market is increasing. Nikon will end production of all lenses for its large format cameras, as well as most of its film camera bodies, interchangeable manual focus lenses and other related accessories. But it will continue to make the F6 film camera in a commitment to professional photographers.

Konica Minolta Exits Camera Market. Terry Sullivan. PC Magazine. 19 January 2006. http://www.pcmag.com/article2/0%2C1895%2C1912898%2C00.asp Konica Minolta, a company that makes cameras, film, and other photographic products, has announced that they are closing their camera and photo divisions. The digital camera assets will be transferred to Sony. Instead, it will focus on copiers, fax machines, printers as well as its medical and graphic imaging technologies. Konica Minolta has been the third largest film maker in the world.


New perpendicular recording boosts hard-drive capacity. Computerworld. January 18, 2006. http://www.computerworld.com/hardwaretopics/storage/story/0,10801,107832,00.html?source=NLT_SU&nid=107832 Seagate has started shipping hard drives that use perpendicular recording, where bits of data are stored vertically rather than horizontally. This increases the amount of storage space on the disk. This is a new way of doing things; it changes the way the heads and disks interact with each other. By reducing the number of disks needed, it can also make the computer run cooler, thus extending the life of the components.

Maxell focuses on holographic storage. Colin Barker. CNET News.com. November 28, 2005. http://news.com.com/Maxell+focuses+on+holographic+storage/2100-1015_3-5973868.html Maxell is planning to launch its first holographic storage in September 2006. The removable drive will hold 300GB. The drive, which looks similar to a floppy disk, uses light-sensitive crystals and can use the whole drive to store the information, not just the surface. The disks will cost about $100. In 5 years they plan to expand the capability to 1.6 TB per disk. Maxell is working with InPhase Technologies. http://www.cnet.com.au/desktops/storage/0,39029473,40058607,00.htm


Imation buys Memorex in $330M deal. Elizabeth Montalbano. Computerworld. January 20, 2006. http://www.computerworld.com/hardwaretopics/storage/story/0,10801,107929,00.html?source=NLT_PM&nid=107929 Imation announced a deal to acquire Memorex, a provider of CD and DVD storage products. The companies will integrate their businesses by the end of the year. The Memorex brand will be retained, and Imation hopes to use this to boost its storage business.

Microsoft Rebukes Blu-ray Statements

Microsoft Rebukes Blu-ray Statements

More on the Blu-ray and HD DVD war. "With Microsoft backing HD DVD and Sony backing Blu-ray, its still anybody's guess as to which format will win."

"Microsoft firmly stands behind the HD-DVD format as the best choice for our consumers. As we've said before, we're confident that HD-DVD will bring the excitement of high definition movies to the consumer faster and at a greater value. We believe that consumers have signaled what format they want to enjoy high definition DVD and we're going to follow them down that path. The question of whether Xbox 360 has the flexibility to adapt to consumers' needs is a different issue entirely. Xbox 360 is a future-proofed system -- one that allows us to add features as consumers demand them -- as evidenced by our offering of the HD DVD drive as an accessory."

Preservation Readings 13 January 2006

Do Burned CDs Have a Short Life Span? John Blau. PCWorld. January 10, 2006.

http://www.pcworld.com/resource/article/0,aid,124312,pg,1,RSS,RSS,00.asp

There are many views on how to store the digital materials, but optical discs have a short life span. There are a few things that can extend the life a few years, such as proper handling and storage. Cheap CDs may have a lifespan of 2 years. Hard drives have limitations and are also prone to failure over time. This article suggests using good quality magnetic tapes, which they claim are the superior storage and can last 30 to 100 years. But most important, have an archiving strategy that allows you to migrate to new technologies.

CDs, Lies, and Magnetic Tapes. Larry Medina. Computerworld. January 10 2006.

http://www.computerworld.com/blogs/node/1552

A response to the John Blau article about CD lifespan problems. The original article had little new to add, and was something that records managers have been saying for years. A sensible media migration policy, is needed. It should look at the usage pattern and methods for media storage and consider the value of information stored. Multiple copies and proper handling and storage will help extend the lifespan. But there are more things to consider, such as the format of the data and hardware obsolescence. Tape was stated in the original article as the better alternative, but there are additional problems with tape, such as heat, magnetism, vertical storage, other storage concerns, and it must be retentioned periodically. We need to diligently protect the information and not rely on things to remain the same forever.

HarperCollins to digitize 20,000 of its books. Reuters. December 12, 2005.

http://www.msnbc.msn.com/id/10442686/

The publisher plans to digitize 20,000 of its existing books and the several thousand books it publishes each year, in order to protect author’s rights from mass digitization projects. They intend to allow internet search companies to create an index. It is also an attempt to control the number of digital copies of works. "If publishers don't do this, there are going to be too many digital copies of books out there."

Elsevier covers archiving with Portico deal. Mark Chillingworth. Information World Review. 22 December 2005.

http://www.iwr.co.uk/information-world-review/news/2147864/elsevier-covers-archiving

Elsevier will deposit over 2000 titles from ScienceDirect in a permanent e-journal archive. It is working with Portico to do this. Portico is the electronic journal archiving arm of JSTOR. This project is part of the national preservation program funded through the Library of Congress. Over seven million articles will be deposited in the archive beginning in January 2006. Elsevier continues to work with the National Library of Netherlands in archiving content.

Preservation Readings 6 January 2006

Copyright Issues Relevant to Digital Preservation and Dissemination of Pre-1972 Commercial Sound Recordings by Libraries and Archives. June M. Besek. Commissioned for and sponsored by the National Recording Preservation Board, Library of Congress. December 2005.
http://www.clir.org/pubs/abstract/pub135abst.html http://www.clir.org/pubs/reports/pub135/pub135.pdf

National Recording Preservation Board has been given the charge to study the state of audio preservation in the US. The Library of Congress is directed to create a comprehensive national plan for audio preservation. This 62 page document is part of that process. Before February 15 1972, federal copyright law did not protect sound recordings, though there are other laws in effect. There are various aspects of copyright discussed. US Code 108 allows libraries or archives to make up to three copies

of an unpublished copyrighted work ‘solely for purposes of preservation and security or for deposit for research use in another library or archives’. The work must be in the of the library collection, and the digital copy may not be made available to the public in that format outside the library premises. Other relevant sections include fair use, first sale doctrine, orphaned works, distance education, ephemeral copying, archival preservation, and some state laws. This article also looks at questions such as
· May Libraries Rely on Others to Make Digital Copies?
· Are Collaborative Digital Preservation Projects Permissible?
· Use of Digital Preservation and Replacement Copies
· Interactive, On-Demand Streaming, & Webcasting

On Technological Protection Issues: The Digital Millennium Copyright Act (DMCA) prohibits circumventing a technological measure that “effectively controls access” to a copyrighted work. There is no exception for archiving, nor is there a general “fair use” type exception in the statute. There is an administrative method for handling exceptions. “If an archives has legally defensible reasons for seeking to circumvent access controls, it could seek an exemption pursuant to the rule-making procedure.” “Technological protections are a potential hindrance to certain library preservation and replacement activities and may require a legislative solution.” Preservation efforts are hampered by legal restrictions. How should a library proceed? Some ways are discussed, such as choosing projects where the risk is low, make copies for preservation but not for current use, focus on older recordings, provide an opt-out mechanism, stream recordings only to other libraries or archives.

“The copyright law has historically granted special privileges to libraries and archives to enable preservation of our cultural and intellectual heritage, and there is every reason to believe that it will continue to adapt to preserve these privileges in the digital world, balancing the needs of libraries and archives with the legitimate interests of right holders. A new study group has been formed to consider the exceptions for libraries and archives in the copyright law and to make recommendations by mid-2006 for possible changes to reflect new technologies.”


-

Perspectives on Trustworthy Information. H.M. Gladney. Digital Document Quarterly. December 2005.
http://home.pacbell.net/hgladney/ddq_4_4.htm

For some, digital preservation means “measures to mitigate the deleterious effects of technology obsolescence, media degradation, and fading human memory.” Others appear to have a broader definition. Using a different phrase for different meanings makes it difficult to understand the topic. Cost estimating should include professional accountants because of their expertise with this. Comparing digital preservation costs between different institutions will be difficult because:
· Costing assumptions differ between institutions, such as the cost of labor, how costs are allocated between projects that share resources, etc.
· Comparing the cost of digital preservation with paper preservation may not work because the objectives may be different
· The cost of facilities to store paper are very different from the ability to store digital items
· The “bottom line” number might not be the most important result of cost estimation. In contrast, the organizational and technical insights accumulated in the exercise can have enduring utility.

There is a hesitancy by many to use checklists such as the Audit Guidelines, because most find them too time consuming and tedious. But there are many detailed requirements needed, so there really is no alternative.


There is slow progress toward practical digital preservation systems. The reasons may be partitioning systems into components, and failing to understand existing software and the additions needed. There are over 70 non-commercial repository packages, too many to really for the marketplace. Most commercial packages are not aware of even the leading non-commercial software. “Much of the current work on digital repositories seems wasteful because it reproduces what might be acquired at less expense than its likely development costs.”


-

Your Data At Risk Why you should be worried about preserving electronic records. The National Council on Archives. September 2005.
http://www.ncaonline.org.uk/materials/yourdataatrisk.pdf

This report is a good overview of digital preservation and is aimed at those who direct information management organizations in the UK, but it is useful to all. “Sustainable digital preservation of records successfully requires a strategic direction and policy commitment from the top of the organisation that translates into operational effectiveness below that level.” “Digital preservation is about a series of actions that need to be taken and managed to make sure there is continued access to digital materials for as long as is necessary.” Digital materials are more ephemeral than traditional materials; they need to be cared for and preserved. This is about cultural memory, but also is needed for a healthy democracy. This is an important part of good records management. It ensures that information is authentic and reliable.

More than just a technological need, the main issue is knowing what you have, what to keep, and how to organize it. “You can’t preserve a record without managing it first.” Preservation requires strategy, structure, processes and responsibility for the entire organization. In addition, it needs clear communication and collaboration from the point of creation. Digital reservation needs the skills of records managers and archivists to select, identify and manage records. They must be involved in the process at the outset—this is critical. The costs to preserve the record cannot be separated from the cost to make it accessible. These costs are an investment and will save money in the long run.
· Developing a Digital Preservation Strategy and a budget
· Identifying a senior management team champion

-

Broadcom unveils chip that plays Blu-Ray, HD-DVD. Tom Krazit. Computerworld. January 03, 2006.
http://www.computerworld.com/hardwaretopics/storage/story/0,10801,107441,00.html?source=NLT_SU&nid=107441

To avoid incompatibilities, Broadcom Corp. has introduced a chip with technology will allow PCs or DVD players to play video recorded with either the Blu-ray or HD-DVD standard. The chip will support the H.264 and VC-1 recording standards, used by Blu-Ray and HD DVD, as well as the high-definition MPEG-2 format.