Living Web Archives (LIWA)

Acronym of the case:

LiWA

Web address of the case:

http://www.liwa-project.eu/

Country of the case:

Czech Republic , France , Germany , Hungary , Netherlands , United Kingdom

Posting Date:

2 July 2010

Last Edited Date:

05 July 2010

Author:

Thomas Risse (L3S Research Center)

web content | digital preservation | web archiving

Type of initiative

Project or service

Case Abstract

The Web today plays a crucial role in our information society: it provides information and services for seemingly all domains, it reflects all types of events, opinions, and developments within society, science, politics, environment, business, etc.Â Due to the central role the World Wide Web plays in today's life, its continuous growth, and its change rate, adequate Web archiving has become a cultural necessity in preserving knowledge. Consequently a strong growing interest in Web archiving library and archival organizations as well as emerging industrial services can be observed.

However, web preservation is a very challenging task. In addition to the â€œusualâ€ challenges of digital preservation (media decay, technological obsolescence, authenticity and integrity issues, etc.), web preservation has its own unique difficulties:

distribution and temporal properties of online content, with unpredictable aspects such as transient unavailability,

rapidly evolving publishing and encoding technologies, which challenge the ability to capture web content in an authentic and meaningful way that guarantees long-term preservation and interpretability,

the huge number of actors (organizations and individuals) contributing to the web, and the wide variety of needs that web content preservation will have to serve.

The aim of the European funded project LiWA (IST FP7 216267) is to create innovative methods and services for Web content capture, preservation, analysis and enrichment.

Project Objectives

The LiWA project, started in February 2008 and led by the L3S Research Center at the Gottfried Wilhelm Leibniz University Hannover, brings together a consortium of highly qualified researchers (L3S, Max Planck Society, Hungary Academy of Science), industrial users (European Archive, Hanzo Archives), and archiving organizations (Sound and Vision, National Library of the Czech Republic, Moravian Library). It is the intention of the project partners to turn Web archives from pure Web page storages into â€œLiving Web Archivesâ€. In order to create Living Web Archives, the LiWA project will address R&D challenges in the three areas: Archive Fidelity, Archive coherence and Archive interpretability:

Archive Fidelity: development of effective approaches and methods for capturing all types of Web content including the Hidden and Social Web content, for detecting capturing traps as well as for filtering out Web spam and other types of noise in the Web capturing process.

Archive Coherence:Â development of methods for dealing with issues of temporal Web archive construction, for identifying, analysing and repairing temporal gaps as well as methods for enablingÂ consistent Web archive federation for fostering synergies between Web archiving stakeholders;

Archive Interpretability: development of methods for ensuring the accessibility, and long-term usability of Web archives, especially taking into account evolution in terminology and conceptualization of a domain

Description of the case

Domain

eGovernment

Topic

Sector

Communication (infrastructure) | Culture and Media | Education, Science and Research

Date

February 2008 to January 2011

Date operational

February 2008

Target Users

Scope

International | Pan-European

Status

Operation

Language(s)

English

Policy Context and Legal Framework

Project Size and Implementation

Type of initiative

IT infrastructures and products

Overall Implementation approach

Partnerships between administration and/or private sector and/or non-profit sector

Technology choice

Standards-based technology | Mainly (or only) open standards | Open source software

Funding source

Public funding EU

Project size

Implementation: €1,000,000-5,000,000

Technology solution

Applications

Two applications scenarios are developed in LiWA to illustrate the possible use of these technologies in real world scenario whose scope is wider than what LiWA specifically addresses.

LiWA technology for content and context in Sound and Vision archive

For audio-visual archives the Web is regarded as a valuable source for gathering contextual information that relates to their audio-visual collections. Context information is relevant for both documentalists, and also other users interested in a specific broadcast or a broadcasting related topic, such as journalists, teachers or researchers. Typically, these users have to use different interfaces for different sources to search these sources. Ideally, Sound and Vision provides these users with a single interface that allows searching both the digital asset management system of the AV archive (iMMix) and related web content. The LiWA application Streaming demonstrates how potential end users (both broadcast professionals and the general public) could access broadcast related web content. In addition, the archived content will be used as test data for the development of the Sound and Vision context data platform that specifically addresses the linking of web context to the digital asset management system of Sound and Vision. Starting off from a mock-up created in the first phase of the project, a first working prototype of the application allows users to search a archived broadcast related webpages and play-back audiovisual content based on a limited number of crawls.

Social web application

Social web sites typically contain highly inter-linked content and use dynamic linking, widgets and tools as well as high degree of personalisation. Capturing social web sites is extremely challenging and cannot be fully achieved using current methods and tools. Social web thus represents one of the greatest challenges in web archiving.

The aim of the application is to show how the LiWA technology fits in the workflow of an active Web archiving institution, by considering a real-life scenario of the National Library of the Czech Republic. The application is designed as a set of independent modules developed in LiWA as described. The modules can be readily integrated with existing Web archiving workflow management tools. A Web archiving institution can choose to deploy all of the modules or just some of them, depending on its needs and particular workflow. The application is designed as generic and can be used to enhance archiving of any type of web content, not just social web.