Trust Through Self Assessment

By Andrew McHugh, The University of Glasgow

Published: 24 October 2007

Browse the paper below or download the pdf.

• Introduction
• The Digital Repository Audit Method Based On Risk Assessment
• Roles and Responsibilities
• Short-term Benefits of the Self-Audit Process
• Longer-term Benefits of the Self-Audit Process
• On the Value of Self-Assessment
• On the Significance of a Bottom-Up Approach
• Additional Resources

1. Introduction

Subject-based and institutional digital repositories are increasingly being hailed as the preferred means for safeguarding the future accessibility of digital information. Repositories will not only store digital materials, but also act to limit the threats that are posed to their authenticity, integrity and understandability over time. Consequently, they are being increasingly relied upon to facilitate curation for a wide range of digital content. Defined more broadly than in mere technological terms, repositories should encompass associated systems, policies, procedures and people and must be considered in terms of the organisational, financial, legal and technological contexts within which digital content is collected, managed, accessed and utilised. Regrettably, within what remains an immature discipline, there are few available assurances of the effectiveness of repositories' adopted approaches, and little evidence of a consensus on what success for digital repositories actually means. The fundamental question is often expressed in terms of trustworthiness; of the repositories claiming to offer information preservation or curation services, which ones can actually be trusted to do so? Information creators, curators and users, as well as funding bodies and other external agencies each have a vested interest in determining the answer to this question.

2. The Digital Repository Audit Method Based On Risk Assessment

DRAMBORA (Digital Repository Audit Method Based on Risk Assessment) is a toolkit that was developed jointly by the DCC and DigitalPreservationEurope   following a series of pilot repository audits undertaken by the DCC in 2006 and 2007. It presents a methodology for self-assessment that encourages organisations to establish a comprehensive self-awareness of their objectives, activities and assets before identifying, assessing and managing the risks implicit within their organisation. Its fundamental aim is to help institutions to create, aggregate and present evidence that demonstrates that they are equipped to act as stewards for our digital resources.

3. Roles and Responsibilities

The purpose of the DRAMBORA toolkit is to help auditors:

• Define the mandate and scope of functions of the repository
• Identify the activities and assets of the repository
• Identify the risks and vulnerabilities associated with the mandate, activities and assets
• Assess and calculate the risks
• Define risk management measures
• Report on the self-audit

DRAMBORA is designed to guide the auditor through the self-assessment process, using a similar analytical method to that which an external auditor would use to examine the repository's activities.

Three principle applications have been identified for DRAMBORA:

1. Validation: using the tool to retrospectively assess the effectiveness of established repository infrastructures in order to plan for their ongoing improvement
2. Preparation: using the tool, with its emphases on aggregation and creation of documentation and risk assessment, to prepare an organisation to welcome external auditors within its environment
3. Anticipation: using the tool to plan repository development and functionality, and identify potential shortcomings or gaps

DRAMBORA promotes a bottom-up approach to repository assessment, and is not tied to any formal benchmarking standard. The organisation itself sets the scope of the audit and the benchmark against which it will be assessed. Nevertheless, DRAMBORA can be used extremely successfully in association with existing objective criteria such as the Trustworthy Repositories Audit & Certification: Criteria and Checklist (TRAC) or the nestor Catalogue of Criteria for Trusted Repositories.

4. Short-term Benefits of the Self-Audit Process

Organisations that complete the DRAMBORA self-assessment process will:

• Establish a comprehensive and documented self-awareness of their mission, aims and objectives, and of intrinsic activities and assets. By defining their operational contexts organisations are well positioned to determine their own assessment parameters as well as verify that their resources are optimally invested and positioned to ensure success.
• Construct a detailed catalogue of pertinent risks, categorised according to type and inter-risk relationships, and fully described in terms of ownership, probability and potential impact of each risk.
• Develop an internal understanding of the successes and shortcomings of the organisation, enabling it to effectively allocate or redirect resources to meet the most pressing issues of concern.

5. Longer-term Benefits of the Self-Audit Process

Organisations that complete the DRAMBORA self-assessment process will:

• Prepare themselves for subsequent external audit whether based upon the Trustworthy Repositories Audit & Certification (TRAC), nestor Catalogue of Criteria for Trusted Repositories, or forthcoming Consultative Committee for Space Data Systems (CCSDS) digital repository audit assessment criteria.
• Demonstrate a commitment to ensuring that the repository is well-equipped to care for digital resources over time.

6. On the Value of Self-Assessment

"By undergoing an examination of their processes, infrastructure, and information-management competence, institutions, information holders, and service providers can obtain a trusted, certified status that provides a sense of reassurance to their various stakeholders. Conversely, where practices are of insufficient quality, audits can highlight this. Publishing the outcomes at least internally can be used to promote higher standards or to alert potential users to shortcomings."

— S. Ross; A. McHugh (2005). "Audit and Certification: Creating a Mandate for the Digital Curation Centre" , RLG DigiNews, Vol. 9 No. 5, ISSN 1093-5371.

7. On the Significance of a Bottom-Up Approach

"What constitutes success in a digital repository can only be addressed in context, specifically the context of the purpose the repository serves and of the environment in which it operates. No repository can be said to be truly successful in a meaningful sense unless it fulfils its purpose. Thus, criteria for success must be derived from its statement of purpose. Similarly, the metrics for estimating success against the criteria must be formulated in light of the culture, constraints and opportunities existing in the environment."

— Kenneth Thibodeau (2007). "If You Build it, Will it Fly? Criteria for Success in a Digital Repository" , Journal of Digital Information, Vol. 8, No. 2, Texas Digital Library.

6. Additional Resources

• Digital Curation Centre (DCC) and DigitalPreservationEurope (DPE), DRAMBORA - Digital Repository Audit Method Based on Risk Assessment (March 2007)
• Center for Research Libraries and RLG OCLC Programs, Trustworthy Repositories Audit & Certification (TRAC): Criteria and Checklist Version 1.0 (February 2007)
• Dobratz, Suzanne; Schoger, Astrid (2005). "Digital Repository Certification: A Report from Germany" , DINI/nestor
• Electronic Publishing Working Group, DINI-Certificate Document and Publications Services 2007 , Draft Version, Version 2 (September 2006)
• nestor Working Group, Catalogue of Criteria for Trusted Digital Repositories, Version 1 (draft for public comment) (December 2006)
• RLG OCLC, Trusted Digital Repositories: Attributes and Responsibilities, An RLG-OCLC Report (May 2002)
• Ross, Seamus; McHugh, Andrew. "The Role of Evidence in Establishing Trust in Repositories" , D-Lib Magazine, Vol. 12, No. 7/8, July/August 2006