National Digital Library Federation (NDLF)
The NDLF Planning Task Force presented this Final Report to the NDLF Policy Board for discussion and planning purposes. The Policy Board accepted the report and has begun to develop the needed strategies and details to carry out the work.
The Policy Board of the National Digital Library Federation recognized in its meeting on October 2, 1995, that the construction of a national digital library that focuses on federation must respect and accommodate locally driven decision making at each institution, while at the same time identifying and endorsing those processes and standards that are the prerequisites for a coherent network of scholarly information resources and services. The Planning Task Force has found in its subsequent deliberations that an organization founded on these principles of federation is not only feasible and compelling, but imperative to insure the affordable and beneficial use of digital technologies by the higher education community.
Much of the technology that would facilitate a federated approach to a national digital library is either already available or well advanced in the process of development. Most of the charter members of the Federation are already working on effective digital services. However, if the individual work is to contribute to a greater whole -- to the construction of a national digital library -- it will need to be based on a set of common structures and protocols. The Planning Task Force has identified three important areas where the research library community can and should exert leadership. The Task Force recommends that the National Digital Library Federation organize itself to pursue them:
If the Policy Board agrees to take on the responsibilities identified by the Planning Task Force, it will need to create a suitable governance structure and an organization capable of propelling the Federation forward. Participating institutions will need to provide at least two types of resources: substantial commitment of local talent and sufficient funding for a variety of key activities, including a full-time program officer. Such an officer could be housed in an existing organization, such as the Commission on Preservation and Access, which would provide office space, support staff help, and similar overhead items. The Task Force estimates total annual costs to range between $150,000-$250,000 for a period of approximately five years, depending upon the number of participants and the capacity to leverage foundation and corporate support.
The targeted areas of business that the Planning Task Force has recommended for the Federation cut across the functional boundaries of traditional library organizations and reflect the broader organizational challenges that research libraries face as they move to provide service for digital materials. In confronting these organizational dilemmas, the Planning Task Force has posed for itself and the Policy Board a fundamental question: Can our institutions afford the necessary investment and organizational transformations to meet future needs in the absence of a federated environment for digital information? The Task Force has answered that we cannot and urges the adoption of the program advanced in this report.
As the nation's cultural resources become increasingly represented in digital form, the need grows to collect, preserve and provide broad access to those resources in ways that are both efficient and affordable. The concept of building a national digital library captures our aspiration to meet this growing need. However, we cannot expect any single institution to serve as the repository for the nation's cultural resources in digital form. Nor, if we posit a system of independent, distributed repositories, can we expect them to manage the nation's digital heritage to standards of broad accessibility, efficiency and affordability without also positing a level of collective action. The National Digital Library Federation aims to define, promote and encourage the development of the necessary means for the nation's digital libraries to act collectively and bring together, or "federate," in the national interest the digital heritage they manage.
Among the many and varied efforts, both individual and collaborative, that are presently underway to build digital libraries across the nation, the National Digital Library Federation is not a membership organization, a consortium or a buying club. Rather, it is a union of institutions that are committed to a particular kind of leadership in the digital arena. The distinctive quality of the National Digital Library Federation is its focus on creating the means -- the common infrastructure -- to federate the digital libraries of participating institutions so that they both provide better local service and advance the common good. The Planning Task Force of the Federation has identified three broad areas in which the development of infrastructure in the form of common standards, protocols and practices will materially advance the ability of digital libraries to extend access to and federate their cultural resources. Development is needed in the areas of discovery and retrieval, intellectual property rights and economic models, and archiving.
Other kinds of technical, legal, organizational and economic development are also needed and underway in many other arenas that are essential to the life of digital information. However, the three areas that the Planning Task Force has identified bear particularly on the goal of federation, and the institutions of the National Digital Library Federation have sufficient interest and expertise in them that they can effectively influence future development. The Planning Task Force has, in fact, charted broadly a course of development in each of these three areas for the Federation and it proposes a series of projects designed to advance that development. In what follows, the Task Force elaborates the rationale for the National Digital Library Federation, defines the three focal areas it has identified as the business of the Federation, and suggests some of the organizational requirements needed to carry out this business over the next five years.
As the Policy Board of the National Digital Library Federation recognized in its meeting on October 2, 1995, an approach to the construction of a national digital library that focuses on federation must respect and accommodate locally driven decision making at each institution, while at the same time identifying and endorsing those processes and standards that are the prerequisites for a coherent network of scholarly information resources and services. The Planning Task Force has found in its subsequent deliberations that an organization founded on these principles of federation is both feasible and compelling.
Much of the technology that would facilitate a federated approach to a national digital library is either already available or well advanced in the process of development. It is the conviction of the Task Force that through the effective application of such technology libraries can and will provide significantly improved services for local users. Most of the research libraries who are charter members of the Federation are already working on effective digital services. However, without an organizational mechanism for them to take collective responsibility for the development and maintenance of a common infrastructure, individual institutions will proceed with efforts that ultimately create disconnected islands within the sea of networked information resources -- projects which are locally useful, but which do not necessarily contribute or build toward a broader resource base and which miss opportunities to take advantage of economies of scale inherent in digital technologies.
The Planning Task Force also sees other reasons that oblige the Federation to move quickly to put in place an effective national infrastructure for scholarly information services. The rapidly growing digital arena has heightened the demand among scholars and the public at large for information resources and services in digital form; this increased demand has in turn spurred aggressive development within the commercial and computer science communities. Research libraries could enhance the emerging digital products if they could find ways to build their expertise and values more effectively into the process of product development. If it is appropriately organized, the National Digital Library Federation can provide a means for research libraries to apply combined economic and political leverage in the broader development process, enabling them to play an influential role in the provision of digital services.
At the same time, the Planning Task Force recognizes that the research library community cannot undertake unilaterally the work of building a national digital library. A well-designed and operating information infrastructure will require close coordination among academic institutions and other information service providers. Digital library collections will comprise components from a variety of heterogeneous systems operated by a variety of independent institutions, including publishers, value-added distributors, scholarly organizations, academic departments and study centers on our campuses, but also (most critically from the standpoint of the Federation) other research libraries. Each of these various institutions will build their digital services based on local priorities and capabilities. However, if the individual work is to contribute to a greater whole -- to the construction of a national digital library -- it will need to be based on a set of common structures and protocols, which in at least three areas it should be the business of the National Digital Library Federation to identify and promulgate.
The Planning Task Force has accomplished its work this year through working groups that explored a variety of technical issues. Some of this work did not bear fruit and resulted in a decision not to proceed further at this time, or to leave the development work to others who are already working on them and better equipped to do so. However, the work of the Task Force was fruitful in three key areas of digital library infrastructure: discovery and retrieval, rights and economic models, and archiving. The research library community can and should exert leadership in these areas and the Task Force recommends that the National Digital Library Federation organize itself to pursue them.
Discovery and Retrieval. The heterogeneity of the information available in digital form -- different data structures, search engines, vocabularies for access -- significantly challenge users in their ability to identify and retrieve needed information. To lower the barriers to access for these heterogeneous materials and to provide cross-collection search capability, the Task Force has charted a multi-step course of action. First, a pilot effort is underway to build a model for "institutional gateways" to digital collections which will allow the aggregation and browsing of digital information by categories of material (e.g., journals, special collection materials, spatial data, etc.). These pilot gateways are currently accessible via the Federation home page on the World Wide Web. Second, to build on this initial step, the Federation should explore adding functionality to the World Wide Web gateways through the incorporation of Internet indexing tools, such as Harvest or Open Text's web index, for the web space of Federation participants. Third, the Federation needs to develop more formal database support for cross-collection search capability. There are a variety of possible database solutions, including the use of SGML, but perhaps the most important issue is for Federation institutions to agree to guidelines for the use of a minimal set of metadata elements in a portable form. These metadata should build on existing efforts, such as the development of the Dublin Core. The elements need to be mapped to MARC or other record formats as desired. They must incorporate naming conventions for digital objects, and they should include descriptive attributes for other infrastructure elements, such as rights and archival status. The Planning Task Force has tentatively formed and charged a working group that could proceed under Federation auspices to address these critically important issues.
Intellectual Property Rights and Economic Models. Since most of the technical requirements for the management of intellectual property rights are now -- or will shortly become -- available, the Federation should concentrate on putting in place a clear and articulate rights policy to regulate rights relationships among Federation institutions. Such a policy will have the effect of organizing common access to digital objects and create incentives for institutions to make digital objects they hold readily accessible via the Federation infrastructure. The principles that the Task Force has considered to underlie an effective rights policy and which it recommends for further investigation and testing include the following:
A further element of an articulate rights policy is how rights to intellectual property affect the economic relationships needed to support the creation, accessibility and maintenance of content in digital form. The Planning Task Force sees the development of the National Digital Library Federation as an opportunity broadly to define and begin to resolve the issues involved in the interaction of rights management and economic organization. The economic issues that most affect the possible future of the Federation and which interact directly with the principles of rights management include the need to establish models of collaborative funding within the research and learning community; the need to create pools of investment capital to support the development of content, access structures and preservation mechanisms; the need to define and rationalize the costs of digital access and preservation; and the need to create revenue streams that recoup development costs, cover ongoing costs and provide incentives for institutions to share and distribute content.
Archiving of Digital Information. Perhaps the greatest test of adherence to the goal of creating a national digital library is a commitment to preserve culturally significant digital information as part of the national heritage. The Federation can foster and facilitate a commitment to digital archiving among participating institutions in at least three ways. First, it can assist in the development of the legal foundations for digital archiving. In the development of its rights management policy, the Federation can help ensure that a common feature of purchase agreements and licenses for digital information is clarity about whether the information will be archived and which party has archival rights and responsibilities. The Federation should also have an interest in defining the archival fail-safe mechanism for which the Task Force on Archiving of Digital Information called in its recently issued final report. Second, the Federation can encourage digital archiving by providing, in its metadata work, a mechanism and clear guidelines for institutions to declare the level of commitment to archiving the material they have made available through Federation means. Third, the Federation can recognize that the prospects of migrating digital information into the future are today more promising and economical for some kinds of materials than for others. To the extent that the Federation helps institutions discriminate among digital materials by the ability to migrate them and develops corresponding guidelines and best practices for digital materials, then it assists in the creation of a trustworthy or, in the words of the report of the Task Force on Archiving of Digital Information, a certified process for preserving digital information.
The Planning Task Force of the National Digital Library Federation began to explore each of these areas of business and found, despite the progress it made, that a great deal more work needs to be done in each. The kind of additional work needed varies, however. For some of the work, like the development of metadata standards for discovery and retrieval or the work on digital archiving, the Federation needs to support the operation of focused task forces or a series of concentrated workshops. For other work, like that on the best practices for licensing digital information, which the Commission on Preservation and Access recently contracted Ann Okerson of Yale University to study, the Federation needs to be able to commission specialized studies and position papers. Finally, in order to create a testbed for development of the three areas of business identified above, the Federation needs to be able to sponsor special projects which serve both to build useful digital content for local and national communities and to engage Federation institutions in focused, collaborative activity. Four opportunities for multi-institution projects have emerged in the deliberations of the Planning Task Force: the National Digital Library project organized at the Library of Congress, the Making of America Project framework initiated by Cornell and Michigan and a Social Science Data effort for which planning is underway within five institutions led by the University of California at Berkeley, and including Stanford and Harvard from among the charter members of the Federation.
If the Policy Board of the National Digital Library Federation agrees to take on the business that the Planning Task Force has identified, it will need to create a suitable governance structure and an organization capable of propelling the Federation forward. Among the organizational requirements for continuing the Federation effort as outlined above, participating institutions need to provide at least two types of resources. First, they need to commit the scarce time of key individuals. This commitment must exceed the level of time that has been available to date for investment in the work of the Planning Task Force. The Task Force recognizes that the assignment of local staffing resources to the support of the Federation will be in some ways even more operationally problematic and politically risky than a simple allocation of funding. However, the real prospects of success for the enterprise rest in no small degree with our willingness to reallocate valuable staff time to cooperative activities and thereby to make an investment that promises a significant return to the local institution as staff from within the Federation share expertise, best practices and new developments.
Second, Federation institutions must generate funding for a variety of key activities such as travel and meeting support, consultants and commissioned papers, and support for electronic communications for sharing information about project activity in digital libraries and for recording and distributing guidelines and best practices. Perhaps most important, in the view of the Task Force, is the need for funding to support a dedicated program officer to push forward the work of the Federation. The Task Force has assumed that such an officer would be housed in an existing organization such as the Commission on Preservation and Access or perhaps the Coalition for Networked Information, and that the host institution would provide office space, support staff help, and other such overhead items.
The Task Force envisions the Federation operating at this modest level of investment for a period of approximately five years. It estimates the total annual costs to range between $150,000 to $250,000 depending on the ambitions of participating institutions for the Federation and the willingness of institutions to shoulder travel and meeting costs without formal reimbursement. The share of the costs that each participating institution needs to bear in this initial phase would depend on the number of institutions invited to participate and on whether or not institutional contributions can be leveraged to raise corporate or foundation support.
Go to:
Digital Library Federation