63rd IFLA General Conference - Conference Programme and Proceedings - August 31- September 5, 1997

Measurement of Electronic Services in Libraries: Statistics for the Digital Age

1. Introduction

How should libraries measure electronic services and media? How should libraries be tracking Internet/World Wide Web usage? What is needed to support library access to electronic information? What defines an electronic session? Should libraries be counting "hits" or "clicks" each time a patron accesses a file, visits a Web site, displays an image, or downloads a page or a document? What statistics should libraries collect on electronic access to remote resources not owned by the library? How are technology and telecommunications expenditures related to electronic media and services affecting libraries? Can conventional library statistical concepts, standards, categories, definitions, and data elements be extended to include electronic media and network services? Will library statistics on electronic services facilitate development of output measures and performance indicators? These difficult and troublesome statistical questions result from the rapid adoption and integration of electronic media and services by all types of libraries.

Conventional library statistics address a wide range of purposes and functions: administration and management; analysis and planning; policy development; and research. Conventional data about library activities and information services are collected and reported at the institutional, state, regional, national, and international levels. These library descriptive statistics reflect varying levels of comparability, consistency, and comprehensiveness. However, agreement about timely, reliable, and relevant statistical information regarding electronic media resources and services in libraries has yet to emerge. This paper raises questions and explores issues related to library measurement of electronic media and network services. Investigation of statistics describing electronic information technologies are critical for organizational efficiency, effectiveness, and performance.

2. Difficulties in Measurement of Electronic Services in Libraries

This paper results from work by the U.S. National Commission on Libraries and Information Science (NCLIS) since 1994. Although focused primarily on public library Internet connectivity [3], this recent Commission work considers general issues about library measurement of electronic media and network services. These issues include the following difficulties in the measurement of these media and services:

• Difficulties in developing standard definitions for electronic media and services related to costs/expenditures for the following:
  • System/server hardware - local integrated systems, terminals, desktop computers, servers, printers, and scanners
    
  • Communications services - telecommunications lines, cabling, routers, modems, network service provider fees, and local area network (LAN) charges
    
  • Software - operating systems, site licenses, applications software, new releases/upgrades, user authentication and validation, and blocking/filtering software
    
  • Training and education - staff training, user education, documentation, and user support services
    
  • Facility upgrades/maintenance - building renovation, cabling and wiring, and equipment
    
  • Content/resource development - collection development, formatting, graphics development, site design and maintenance, and commercial arrangements
    
  • Program planning/management/staffing - staff recruitment, budget preparation, program analysis, planning, and consultation
• Difficulties in developing standard definitions for measuring the impact of collaborative and cooperative activities between/among libraries and other institutions in providing electronic media and services
  
• Difficulties in accommodating successive new generations of systems and software release changes, and in addressing obsolescence
  
• The open and unrestricted nature of network services makes control, regulation, and planning difficult
  
• Difficulties resulting from complex telecommunications infrastructure technologies and rapidly changing bandwidth pricing structures
  
• The challenge of keeping current with rapid pace of change in business and pricing practices from electronic content publishing industry and network service providers
  
• Confusion and uncertainty relating to document delivery services and "fair use" of copyrighted material in electronic format
  
• Lack of standards for quantitative measurement of electronic multi-media works or objects, or for derivative works
  
• Difficulties in relating conventional transaction-based library statistics to interactive electronic networking activities

In general, the measurement of electronic media and services presents significant challenges for libraries and the evolving information services field. Standard statistical methodology, definitions, structures, and categories for describing digital media resources and services are essential for library management and planning. The rapid pace of technological change, together with the early adoption of electronic information and digital communications technologies by the global library community add to the critical importance of these concerns. To manage the changes resulting from these electronic and networking technologies, standard measures and descriptive statistics need to be formulated. The future relevance and viability of the library's mission in the emerging age of digital information requires that these difficult issues be addressed.

3. Approaches to Electronic Information Service Measurement

It is hard to overestimate the impact that information technology has had on libraries in the last several decades. At the same time, however, the impact of this rapid technology adoption challenges conventional library statistics and measurement concepts. Simple extension of conventional descriptive statistical structures relating to libraries raises difficulties that seem to require the development of new measurements based on electronic commerce than from library statistical standards. These new electronic service offerings may, at a fundamental level, require the reconceptualization of library quantitative measurement.

Such a reconceptualization may involve measurement of information technology-based media and services in libraries based on a combination of approaches such as the following:

1. Transaction-based measures - where interactive sessions, downloads, hits, terminals/patron, domain and host addresses, images, or files are counted, recorded, and measured by sampling or by transaction logs
   
2. Time-based measures - where available service hours, session length/duration, system/server peak level are measured and reported
   
3. Cost-based measures - where measures are made based on cost/expenditure for telecommunications/bandwidth, terminal/hardware equipment, staff, training, maintenance, site licenses
   
4. Use-based measures - where user activities, anticipated demand, simultaneous users, group use, hits/patron, user satisfaction, local or remote/off-site use is measured

These approaches for measuring electronic services can be compared with the following trends and requirements:

• Efforts to define and measure remote access and use of local electronic media resources in libraries are matched with efforts to identify and track local use of remote resources
  
• Network log analysis software and Web-counters are available which provide information about domain and host Internet traffic, but it is uncertain how such commercial program reports can be used by libraries in measuring electronic media and services
  
• Libraries are increasingly offering access to metadata to help users to identify/discover electronic media resources and services, but at present, there appear to be no standards for measuring or reporting metadata (or OPAC) availability and usage
  
• The traditional library measurement orientation and focus on acquisitions and collection ownership appears to be shifting towards the need for information about library users and especially on statistics related to their service needs
  
• Electronic information media and resources are increasingly used in disintermediated settings where users have little contact or interaction with library staff
  
• There is a trend toward libraries serving as the point-of-contact for locally mounted electronic media resources accessible from different sites on campus or from branches
  
• The electronic environment is reducing traditional differences and distinctions between types of libraries, and between libraries and other information-based organizations
  
• Negotiating site licenses with electronic information service providers requires that the library provide the anticipated number of potential simultaneous users for a particular resource or database
  
• With no standards for the number of terminals required per user, increased demand for access to electronic media and network services has forced some libraries to employ sign-up/queueing sheets to allocate access in limited-time increments
  
• Libraries are creating navigational or finding aids by "data-mining" activities that establish electronic links to and among digital and print resources, but it is difficult to measure this activity or the effectiveness of these investments

4. Standards for Measurement of Electronic Information Services in Libraries

This paper began by posing a series of difficult and troublesome questions related to the adoption of electronic media and services in libraries. In many ways, these questions offer an opportunity to re-examine basic library statistical concepts. The questions reflect the pervasive influence of electronic information technology on all aspects of library operation and measurement. Statistics about library collections, services, finances, staffing, usage, and transactions are influenced by increased library involvement with electronic media and services. But, along with the challenges, appears an inherent promise for measuring the needs of library users.

Internet/WWW use logs offer the potential for identifying local and remote users of electronic media and services through system-generated information that libraries could employ to track and profile users and their specific needs. This type of information can involve the capture of new data about who uses what resources, but it may also involve libraries in sensitive issues related to patron confidentiality. While conventional library statistics about patron behavior provide data that reflects use of collections and services, electronic measurement of library use offers more detailed information about customers of specific electronic media resources and services not owned or provided by the library. Additionally, measurement of electronic media and services use offers the potential to identify users' expectations and satisfaction.

While libraries have developed conventional quantitative standards for traditional print-based collections and services, electronic information services may provide libraries with opportunities to gather qualitative information concerning performance effectiveness and outputs. This potential offers a rich arena for libraries. For libraries to respond effectively to demands for greater accountability, evaluation, and efficiency, they require new assessment tools to measure impact. The instruments provided by electronic technologies may supply those tools needed to plan for the future during times of fluid change and uncertainty.

Notes

1. As of June, 1997 Peter R. Young assumed the position of Chief, Cataloging Distribution Service at the Library of Congress, Washington, D.C.
   
2. The views and opinions expressed in this paper are those of the author and do not necessarily reflect the policies or positions of the U.S. National Commission on Libraries and Information Science or the U.S. government.
   
3. U.S. National Commission on Libraries and Information Science. Public Libraries and the Internet: Study Results, Policy Issues, and Recommendations - Final Report. Washington, D.C.; June 1994.
   U.S. National Commission on Libraries and Information Science. Internet Costs and Cost Models for Public Libraries - Final Report. Washington, D.C.; June 1995.
   U.S. National Commission on Libraries and Information Science. The 1996 National Survey of Public Libraries and the Internet: Progress and Issues - Final Report. Washington, D.C.; July 1996.