This is a draft paper by David Bearman explaining the Reference Model for Business Acceptable Communications and attempting to explain why it is important. David has graciously agreed to allow us to link this paper to the Metadata Workshop Web. Please note that the formatting is not authoritative, nor is this a citable text at this time, though I think he would be pleased to hear the reactions of thoughtful reviewers. David Bearman stu ---------------------------------------------------------------------- Virtual Archives David Bearman ICA Meeting, September 1996, Beijing Abstract: In an era of electronic communications, there is less rationale for gathering archives in one physical place than there has traditionally been. Indeed, many arguments of practicality and expense can be raised against planning on physical archives in an electronic age. However, before we can move to completely virtual electronic archives, all electronic records must have associated with them the metadata that gives them their recordness and enables us to manage them appropriately over time. This metadata must be grounded in an understanding of the functional requirements for recordkeeping and the meaning of evidence and it must support all the processes from creation to destruction or access that will need to be applied to the documents. These metadata, called here the metadata of "Business Acceptable Communications" will serve as the foundation for virtual archives and provide us with the ability to re-invent the archives of the future along distributed lines, with great benefits to the archives profession and to researchers. Virtual Archives Introduction Within the next decade, almost all organizational records created in our society will be made and communicated electronically. As a consequence, in as little as a generation, the vast majority of all organizational records ever created will be electronic. Because the location of electronic records has little to do with their accessibility, we will then be able to access any records from anywhere with equal ease. As a consequence, today's physical repositories of archives based on custody, will become nodes in virtual archives which also include records outside archival custody but under archival control. The virtual archives of the future will be maintained not through accessioning, preservation and provision of on-site access, but through the control of information about records, and its use to ensure retention, disposition and access. No new technology is required to achieve this end, all that is necessary is that we adopt standards and methods today that will lead to this result. Achieving this goal would simultaneously reduce the overheads involved in physical care of records and the management of their disposition. Without such standards, the expense of migrating records across software and hardware generations will greatly exceed that of managing our current paper-based archives, but with the adoption of appropriate standards, the marginal cost of on-going management of archival records will be minimal. This paper does not explore the professional and organizational aspects of such reengineering of the archival profession, which have been dealt with elsewhere. 1 Neither does it examine the technical case, or costs, of reconfiguring paper-based archives, because this case is being made in practice by Digital Library projects throughout the world. 2 Rather it focuses on two critical barriers to ubiquitous creation and management of electronic records: First, the assurance that such records satisfy the requirements for evidence and Second, methods by which records can be made available over time without constant re-presentation and migration of their intellectual contents. Unless we can ensure that these two requirements are satisfied in electronic environments, no electronic records systems can be archival. If these two criteria can be satisfied, and we act to implement these methods in all electronic communications systems, all electronic environments will be able to support the functional requirements for recordkeeping and no special recordkeeping systems need be implemented to redundantly hold and provide access to archival records. Archivists are far from alone in seeking to define and implement environments that will ensure the creation and manageability of electronic records. In business arenas from commerce to health care, and from research and development to manufacturing, managers are seeking to define standards for data interchange adequate for their business purposes because such electronic systems will make their businesses more effective. The literature is replete with jargon laden discussions of how to enable end-to- end electronic business interaction to support such new processes as would be enabled by electronic patient records or electronic laboratory notebooks or to satisfy documentation requirements imposed by process oriented quality standards such as CALS or ISO-9000.3 Lawyers and auditors place demands on network designers to ensure that electronic records, satisfying requirements for evidence and manageability, are created in communications environments within and beyond the boundaries of the organization. Already they are are encountering requirements to identify records, control access to them, manage software dependencies which will impact on the usability of records over time, and find ways to represent the contextual significance (business meaning) of the records that have been created.4 Indeed many critical observers have argued that unless we can satisfy requirements for "integrity", "authenticity", "reliability" and "archiving" of digital information, the National and Global Information Infrastructures will never be able to support serious work.5 Archivists have not entirely ignored these emerging requirements.6 At the University of Pittsburgh School of Library and Information Science, faculty and students engaged in a research project funded by the National Historical Publications and Records Commission have been examining the "Functional Requirements for Recordkeeping" and have developed a specification of the attributes of "recordness" or evidentiality.7 The specification defines thirteen properties which are identified in law, regulation and best practices throughout the society as the fundamental properties of records. By formally restating these detailed specifications, the researchers have derived "production rules" or logical statements of simple observable attributes which when present, ensure that a system is creating and maintaining records as evidence.8 At this level of specificity, the production rules, and hence the functional requirements for recordkeeping, can be demonstrated to be satisfied by the presence of specific information about the time, place and business function of the record creation, the ways the data are structured and the content of the communication itself. This specific information about records is called metadata. The functional requirement is satisfied if the metadata essential to record content and structure is inextricably linked to and retained with the metadata defining with the context of the business transaction it enabled. This metadata guarantees that the record will be usable over time, only accessible under the terms and conditions established by its creator, and have properties required to be fully trustworthy for purposes of executing business.9 The metadata requirements for evidentiality or "recordness" constitute a definition of information required for "business acceptable communications". Business acceptable communications could be enabled by a scheme for electronic envelopes containing business communications that would ensure that the envelopes could be opened by different computers in the future and their contents would still be accurate, understandable and meaningful. The metadata required to open such envelopes in the future, if always associated with the record contents, would also ensure the viability of virtual archives. Wherever the record was stored, it would always contain the information required for systems to manage its disposition and/or provide access. In order to guarantee such distributed archival functionality, standards must be adopted which prevent the creation and communication of records that are not business acceptable communications. The need for such standards is widespread. Not only would they make communications received over networks trustworthy for the purposes of conducting business, and help to ensure accountability and protect organizations against the risks of loss of proof of their past behavior, they could, if properly specified, greatly simplify: * the management of huge volumes of communications from different systems, * the proper retention and disposition of records, * the retrieval and use of records, and * the appropriate management of private, secure, proprietary and business confidential data. Since standards which capture record metadata must exist for any system or application, and be implemented at many different layers of software and hardware, it is necessary to define the relationships between these metadata standards and how the comprehensiveness of a given set of standards can be determined.10 Such definitions of relationships between standards are called "reference models" because they serve as reference points for designers to structure the methods they develop to implement the standards. This paper introduces such a reference model, but first, to understand what data is necessary for business acceptable communications, we examine in more detail the nature of electronic evidence. Electronic Evidence and the Functional Requirements for Recordkeeping: Transactions (actions taken 'across' something) are by definition actions communicated from one person to another, from a person to a store of information, such as a filing cabinet or computer database, or from a store of information to a person or a computer.11 As such, transactions must leave the human mind or the computer memory in which they are created and be spoken, written or read from elsewhere. Electronic transactions must be conveyed across at least one software layer, and typically across a number of hardware switches or connections, in order to be communicated. Records are the carriers, products and documentation of transactions. Not all data is a record because not all data completely represents the transaction in which it was engaged. In fact, most information created by and managed in information systems, is not a record and lacks the properties of evidence. Records will only be evidence if the content, structure and context information required to satisfy the functional requirements for recordkeeping is captured, maintained and usable. The functional requirements for capture, maintenance and use of the content, context and structure of records derive their warrant from law, regulation, and professional guidelines.12 The requirements for recordkeeping are corporate requirements, not those of a single business function, and are therefore applicable to any organizational communications. These requirements are the foundation of good business practices and are essential guidelines in reducing risks from increased liabilities or decreased opportunities that accompany poor recordkeeping practices. Records oriented professionals within organizations, such as senior management, legal counsel, auditors, Freedom of Information and Privacy officials, and archivists all require records, and not just information, for their on-going work. Organizations want to satisfy these requirements in the normal course of business, but it has been difficult to do so in the computer based communications environments we have installed in the past because applications software has not created or managed the metadata required by records. Information systems are generally designed to hold timely, non-redundant and manipulable information, while recordkeeping systems store timebound, inviolable and redundant records. Few, if any, in-house information managers have been able to devote the energy to rigorous definition of the distinct requirements for recordkeeping or, if they had, would be able to envision how to satisfy these throughout all systems. Without explicit and testable specifications, these systems have failed to satisfy the requirements for recordkeeping and are, therefore, a growing liability to companies even while they are contributing directly to day-to-day corporate effectiveness. The "Functional Requirements for Recordkeeping"(FRR's)13 dictate the creation of records that are comprehensive, identifiable (bounded), complete (containing content, structure and context), and authentic. These four properties are defined by the FRR's in sufficient detail to permit us to specify what metadata items would need to describe them and to audit systems for these properties. This descriptive metadata must be managed in a way that prevents its being separated from the record or being changed after the record has been created. We can envisage a record as a metadata encapsulated object (content in an envelope with metadata on the outside), although in fact it might not be physically stored in this manner just as we might keep a paper case file separate from the indexes which identify it. When transmitted, the contents of the record are preceded by information identifying the record, the terms for access, the way to open and read it, and the business meaning of the communication. Metadata encapsulated objects may contain other metadata encapsulated objects, because records frequently consist of other records brought together under a new "cover", as when correspondence, reports and results of database projections are forwarded to a management committee for decision. The metadata required to ensure that functional requirements are satisfied must be captured by the overall system through which business is conducted. The system here includes personnel and policy in addition to hardware and software. The metadata created with the record must allow the record to be preserved over time and ensure that it will continue to be usable long after the individuals, computer systems and even information standards under which it was created have ceased to be. Several additional requirements define how the data must be maintained and ultimately how it and other metadata can be used when the record is accessed in the future. What metadata then should encapsulate a data object in our envelopes and how should it be structured? The metadata elements discovered in the analysis of the functional requirements for recordkeeping can be organized in six layers based on the functionality that needsto be supported. Each layer of metadata is composed of information relevant to specific hardware, software and organizational entities organized to do a specific job within the overall recordkeeping and communications environment. These layers are: * Registration * Terms & Conditions * Structure * Context * Content * History of Use We have already mentioned that records may contain other records, and contain data from a variety of sources in any possible data format, so it is evident that the metadata must identify, or "register" a record uniquely. Only in this way can we know what data was in, and out, of a given transaction, and that the data in a communication is being declared to constitute a record. This record registration function is primary, and independent of all other functions of the record, so its metadata constitutes a discrete layer. We also must know if the record is available for reading, and if so to whom and under what authority. For this purpose the encapsulation must next contain a layer of information related to terms and conditions of use. This metadata supports the functions of security control, permission negotiation and payment and sets any tables necessary to invoke redaction of record contents based on privacy, confidentiality or secrecy. Because these functions must be handled before the record itself is released and its subsequent metadata becomes available for analysis by the requestor, this metadata also occupies its own layer and comes next. Ideally all data objects that we want to communicate, that is the contents of all records, would be "interoperable" over a substantial period of time. To be interoperable, it must be possible to open and read them using computer systems other than those which created them. We attempt to achieve such inter-operability by encoding contents in standard formats to give them a degree of software independence, making them usable by software other than that which created them. However, many data objects that we create today cannot be standardized and the actual degree of software independence which can be ensured depends on how long any given "standard" can be expected to remain a standard. Therefore, the metadata with which we encapsulate records must flag the dependencies of the data (including their dependency on standards) so that a future review of the encapsulating metadata or "record headers" can locate potential sources of expiring standards or software dependencies and segregate records requiring logical "re-presentation" (called migration to new software formats) before they become unreadable. We record these dependencies in a layer of structure metadata encountered by the software in which the record will need to be represented. This metadata should enable the remaining information to be opened and meaningfully interpreted. A specification of the metadata required to define the management requirements for evidence of business transactions arising from distinct processes can ensure a reduction of corporate risks and support formally auditing the business system. It enables us to locate where and how software, hardware, procedures and policies surrounding a system contribute, or fail to contribute, to the creation, maintenance and use of evidence. While no system of management can be self-auditing, a communications system built to ensure that appropriate metadata is captured for evidence can support a level of management accountability that it was never previously possible to implement or enforce. For these purposes we need to maintain contextual metadata, contained in a layer that precedes content so that it is always present as provenance. Following the contextual layer, comes the content itself. This is the data which was engaged in the transaction, and it may take any form. Technically this is a BLOB (binary large object) which may contain text, images, sound, and even links to other systems and data in executable form (although such dynamic elements will make the long-term preservation of such records highly problemmatic). Finally, our concept of evidence makes it important to know when records were used and how, in what ways they were filed, classified and restricted in the past. We also need to know if records have been destroyed, when and by whom and under what disposition authority that act took place. It is also important to us to know what redacted versions of records were released over time. Our traditional recordkeeping and archival control systems enable us to document such important events in the lives of records, if at all, only on an aggregate level. Electronic records are communicated each time they engage in a transaction such a filing, classification, destruction or release, therefore transactional data reflecting the history of records use will be generated by electronic recordkeeping systems, but instead of such data residing only at aggregate levels, electronic records metadata structures enable us record this data for each record. An important class of metadata is, therefore, historical transactions data. This data is retained in a layer at the end of the record to allow new transactions to be added to the data stream in the order of their chronological occurence without having to open the record. The Metadata of the Reference Model The individual metadata elements discovered in the analysis of the production rules for the functional requirements for recordkeeping can be clustered by their intellectual commonalities. Elements refering to the same aspects of the record are "clustered" in this way for convenience of reference, use and representation. Each cluster may consists of some mandatory and some optional metadata elements, and sometimes of a potentially extensible set of data elements specific to a business process. The clusters themselves are considered part of the reference model and must always occur. The metadata content directly related to satisfying requirements for evidence is always mandatory. Hence evidence, required for the conduct of business and for accountability, is ensured by a Metadata Encapsulated Object conforming to the reference model for "business acceptable communications". The metadata content which contributes to recordkeeping, or management of records, but is not essential to evidence, is optional. Metadata content useful for specific business functions may be defined as mandatory for business in that domain or optional. The clusters we have identified are: Registration Metadata Layer Record Declaration Cluster Identifier Cluster Terms & Conditions Metadata Layer Rights Clearance Cluster Access Permission Cluster Use Conditions Cluster Retention Cluster Structural Metadata Layer File Identification Cluster File Encoding Cluster File Rendering Cluster Record Rendering Cluster Content Structure Cluster Source Cluster Contextual Metadata Layer Transaction Context Cluster Responsibility Cluster Business Function Cluster Content Layer Content-Description Cluster Record Use History Metadata Layer Use Cluster Within each cluster are one or more metadata elements referencing specific properties of the record. These elements are detailed in the reference model itself but not discussed here in the interest of preserving the focus of this paper on virtual archives. The reference model, at this writing, is a proposal being presented to a variety of standardization bodies and professional community forums.13 Metadata Encapsulated Records and Virtual Archives The reference model sketched above ensures the creation, preservability and deliverability of evidence over time. It also contributes directly to enabling virtual archives. First, because the reference model can guide implementations that result in appropriate metadata even before standards are adopted that guarantee these outcomes. Second, because we are observing a large number of standardization efforts in many parts of the information technology community which are creating standards for metadata encapsulated objects but are as yet uninformed by the Functional Requirements for Recordkeeping and could, as a consequence of the reference model, converge and produce outcomes consistent with creation of records. It is important to note that in addition to ensuring that the data we capture is a record, and can serve as evidence, the metadata requirements established for the reference model are designed to ensure that communicated data objects are: * self-documenting * self-authenticating * self-redacting * self-migrating, and * self-disposing. These highly desirable properties will be imparted by keeping records with metadata conformant to the reference model. In the absence of standards that conform to the reference model, these benefits could be guaranteed by capturing and maintaining this same metadata by design, policy or system implementation practices. Although the reference model introduced here is still only a proposal, there is good reason to expect a model of this sort to prevail soon because if there was conformity in metadata encapsulation it would greatly simplify the engineering of networked, distributed, business communications systems and because if there is not, it will be very difficult to use networked communications for multi-lateral business purposes. Designing networked applications within a framework in which transactions are encapsulated by appropriate definitive metadata, allows us to ignore where records are stored or what specific systems created the records, and to concentrate instead on generic records management functions that locate records which are scheduled for disposition and carry out the appropriate actions, or locate records which require conversion from one standard format to another. On the other hand, even imagining a commercial transaction outside of the assurances of the reference model presumes a willingness on the part of the two parties to a transaction into enter into a specific bilateral contracts, such as those executed in support of EDI (electronic document interchange). Such bi-lateral agreements could not be executed for every possible business transaction, hence electronic systems will ultimately need to adopt standards such as those envisioned by the reference model. Needless to say, bi-lateral arrangements would undermine archival record retention while multi-lateral agreements make electronic recordkeeping possible for the same reasons they allow electronic business. Conclusions Virtual archives, as opposed to physical repositories, will exist when users anywhere can access records anywhere without having to be aware of the source from which they came. Virtual archives will make use of the fact that electronic storage and retrieval of records is most effective when records are accessible from sources near the majority of the users. In a distributed environment, the actual storage location of records can follow patterns of use if all records are managed in a uniform environment and carry with them the knowledge of how they are to be disposed, preserved, and accessed. This knowledge is embodied in metadata that must be kept with the records at all times. If the metadata that is kept with records ensures their recordness or evidentiality, and the metadata is represented in elements, clusters and layers that support recordkeeping management requirements, then virtual archives can succeed. The Reference Model for "business acceptable communication" proposes such a framework for metadata which is grounded in archivally sound principles. Virtual archives can, and will, thrive in an environment in which such a reference model informs standards and implementations of electronic communications systems. Footnotes/Citations 1) David Bearman & Margaret Hedstrom, Reinventing Archives for Electronic Records: Alternative Service Delivery Options" in Hedstrom, Margaret ed. Electronic Records Management Program Strategies, Archives and Museum Informatics Technical Report #18, 1993 p.82-98 Margaret Hedstrom, "Electronic Archives: Integrity and Access in the Networked Environment" (Second International Conference on Scholarship and Technology in the Humanities, January 1994) forthcoming New York State Archives and Records Administration, Building Partnerships for Electronic Recordkeeping: The New York State Information Management Policies and Practices Survey. Summary of Findings (by Margaret Hedstrom, Project Director), August 1994 2) Library of Congress, The National Digital Library Program (Washington DC, January 1995) Research Libraries Group, Digital Imageing Technology for Preservation: Proceedings from an RLG Symposium held March 17 & 18, 1994, edited by Nancy E. Elkington (Mountain View CA, RLG, 1994) Anne R. Kenney & Lynne K. Personius, Joint Study in Digital Preservation: Report Phase I (January 1990-December 1991) (Washington DC, Commission on Preservation and Access, 1992) Anne R. Kenney, Michael A. Friedman and Sue A. Poucher, Preserving Archival Material Through Digital Technology. Final Report (Ithaca NY, Cornell University Library, 1993) 3) A.H.Boss, "Electronic Data Interchange agreements: private contracting towards a global environment", Northwestern Journal of International Law and Business, vol.13 (1992) Short citations to sources that have provided strong evidence of the requirements of specific application domains include: Electronic Data Interchange Association, The United States Electronic Data Interchange Standards J.L.Lamprecht, Implementing the ISO 9000 Series (1993) A.J.Marcella & S. Chan, EDI Security, Control and Audit (1993) Miller, GAAS Guide (1994) J.A. Rabbitt, The ISO 9000 Book: A Global Competorts Guide to Compliance and Certification (1993) J.P. Tomes, Healthcare records management, disclosure and retention (1993) B.Wright, The Law of Electronic Commerce (1991) 4) IEEE Mass Storage Systems Standards Technical Committee Metadata Project, Second Meeting on Metadata for the Administration and Access of Stored Information, Austin Texas February 17-18, 1994 Documents discussed at this meeting included: - "The Intelligent Archive" (UCRL-TB-115079-6 Lawrence Livermore Laboratory, Carol Hunter Project Manager) - "Whitepaper on Data Management", Robyne Sumpter, Lawrence Livermore Laboratory Feb.10, 1994 - "A Metadata Capability Supporting the Hierarchical Storage and Access of Large Abstract Data Entities", J.C.Almond and Rekha Singhal, University of Texas CHPC 5) For example, see: Clifford Lynch, "The Integrity of Digital Information: Mechanics and Definitional Issues", Journal of the American Society for Information Science, vol.45#10, December 1994 p.737-744; Peter Graham, "Intellectual Preservation in the Electronic Environment", Proceedings, Library Collections and Technical Services 1992 pp.18-32 (Chicago, ALA, 1992); Henry Perritt, "Public Information in the National Information Infrastructure", Report to the Regulatory Inforemation Service Center, General Services Administration and to the Administrator, Office of Information and Regulatory Affairs, Office of Management and Budget, 5/20/94 6) New York State Archives & Records Administration, Guidelines for the Legal Acceptance of Public Records in an Emerging Electronic Environment (Albany, Dept.of Education, 1994) 35pp. 7) NHPRC grant #93-030, "Variables in the Satisfaction of Archival Requirements for Electronic Records Management" see: David Bearman et.al., Functional Requirements for Recordkeeping, 1994 Version reprinted here as Appendix A. See also, David Bearman, Electronic Evidence: Strategies for Managing Records in Contemporary Organizations (Pittsburgh, Archives & Museum Informatics, 1994) Richard J. Cox, "Re-Discovering the Archival Mission", Archives and Museum Informatics/Cultural Heritage Informatics Quarterly vol.8#4, 1994 p.279-300 8) David Bearman and Ken Sochats, "Formalizing Functional Requirements for Recordkeeping" unpublished draft paper included in University of Pittsburgh Recordkeeping Functional Requirements Project: Reportts and Woreking Papers (LIS055/LS94001) September 1994 9) David Bearman, Functional Requirements for Recordkeeping: Metadata Specification (Unpublished Draft, 2/21/94) 10) For example, we would want to know whether the metadata specified in the "Datastream for Folder Interchange" (ISO 161/17-WG6 NWI) is adequate to ensure recordkeeping, or if it is essential to establish bi-lateral agreements such as those enabled by the Electronic Document Interchange standards or funds transfer and automated teller system protocols. Does the Spatial Data Interchange Format or the DIGEST specification, tell us all we need to know to reconstruct geographic information system records in a few decades? To answer questions like these we need a formal specification and reference standard. 11) David Bearman, "Electronic Records Management Guidelines: A Manual for Development and Implementation" in United Nations, Administrative Coordinating Committee for Information Systems, Management of Electronic Records: Issues and Guidelines (New York, UN, 1990) reprinted in Electronic Evidence, op.cit.fn5 12) The Functional Requirements for Recordkeeping project has compiled a database of "warrant" for the requirements we have defined. The most up-to-date version of the requirements, specifications, production rules, metadata standards, literary warrant and research papers on the variables in electronic recordkeeping in organizations are maintained on the project WWW server at the University of Pittsburgh. 13) David Bearman, A Reference Model for Business Acceptable Communication, Unpublished proposal, 1995 Appendix A. FUNCTIONAL REQUIREMENTS FOR RECORDKEEPING Organization - Compliant 1. Compliant: Organizations must comply with the legal and administrative requirements for recordkeeping within the jurisdictions in which they operate, and demonstrate awareness of best practices for the industry or business sector to which they belong and the business functions in which they are engaged. 1a) External recordkeeping requirements are known. 1a1) Laws of jurisdictions with authority over the record creating organizations are known. 1a2) Regulatory issuances of entities with administrative authority over the record creating organizations are known. 1a3) Best practices of recordkeeping established by professional and business organizations within the industry and business functions of the organization are known. 1b) Records created by organizational business transactions which are governed by an external recordkeeping requirements are linked to an internal retention rule referencing the documented law, regulation, or statement of best practice. 1c) Laws, regulations, and statements of best practice with requirements for recordkeeping are tracked so that changes to them are reflected in updated internal recordkeeping instructions. Record Keeping Systems - Accountable 2. Responsible: Recordkeeping systems must have accurately documented policies, assigned responsibilities, and formal methodologies for their management. 2a) System policies and procedures are written and changes to them are maintained and current. 2b) A person or office is designated in writing as responsible for satisfying recordkeeping requirements in each system. 2c) System management methods are defined for all routine tasks. 2d) System management methods are defined for events in which the primary system fails. 3. Implemented: Recordkeeping systems must be exclusively employed in the normal course of business. 3a) Business transactions are conducted only through the documented recordkeeping system and its documented exception procedures. 3b) No records can be created in the recordkeeping systems except through execution of a business transaction. 3c) Recordkeeping systems and/or documented exception procedures can be demonstrated to have been operating at all times. 4. Reliable: Recordkeeping systems must process information in a fashion that assures that the records they create are credible. 4a) Identical data processes permitted by the system must produce identical outcomes regardless of the conditions under which they are executed. 4b) Results of executing systems logic are demonstrable outside the system. 4c) All operational failures to execute instructions are reported by the system. 4d) In the event of system failures, processes under way are recovered and re-executed. Records - Captured 5. Comprehensive: Records must be created for all business transactions. 5a) Communications in the conduct of business between two people, between a person and a store of information available to others, and between a source of information and a person, generate a record. 5b) Data interchanged within and between computers under the control of software employed in the conduct of business creates a record when the consequence of the data processing function is to modify records subsequently employed by people in the conduct of business. 6. Identifiable: Records must be bounded by linkage to a transaction which used all the data in the record and only that data. 6a) There exists a discrete record, representing the sum of all communications associated with a business transaction. 6b) All data in the record belongs to the same transaction. 6c) Each record is uniquely identified. 7. Complete: Records must contain the content, structure and context generated by the transaction they document. 7a) Accurate: The content of records must be quality controlled at input to ensure that information in the system correctly reflects what was communicated in the transaction. 7.a1) Data capture practices and system functions ensure that source data is exactly replicated by system or corrected to reflect values established in system authority files. 7.b) Understandable: The relationship between elements of information content must be represented in a way that supports their intended meaning. 7.b1) Meaning conveyed by placement or appearance of data are retained or represented. 7.b2) System defined views or permissions are retained and the effects are reflected in the record are represented. 7.b3) Logical relations defined across physical records are retained or represented. 7.b4) Software functionality invoked by data values in the content of the record are supported or represented. 7.c) Meaningful: The contextual linkages of records must carry information necessary to correctly understand the transactions that created and used them. 7.c1) The business rules for transactions, which minimally locate the transaction within a business function, are maintained. 7.c2) A representation of the source and time of the transaction which generated a record is maintained. 7.c3) Links between records which comprised a business activity are retained. 8. Authentic: An authorized records creator must have originated all records. 8a) All records have creators which are documented. 8b) Records creators must have been authorized to engage in the business transaction that generated the record. 8c) A knowledge-base of persons authorized to engage in business transactions is maintained and either operates as a control over system functions such that transactions could not occur without being authorized and/or documents the authorization of the creator as part of the record. Records - Maintained 9. Preserved: Records must continue to reflect content, structure and context within any systems by which the record are retained over time. 9.a Inviolate: Records are protected from accidental or intended damage or destruction and from any modification. 9.a1) No data within a record may be deleted, altered or lost once the transaction which generated it has occurred. 9.b Coherent: The information content and structure of records must be retained in reconstructable relations. 9.b1) If records are migrated to new software environments, content, structure and context information must be linked to software functionality that preserves their executable connections or representations of their relations must enable humans to reconstruct the relations that pertained in the original software environment. 9.b2) Logical record boundaries must be preserved regardless of physical representations. 9.c Auditable: Record context represents all processes in which records participated. 9.c1) All uses of records are transactions. 9.c2) Transactions which index, classify, schedule, file, view, copy, distribute, or move a record without altering it are documented by audit trails attached to the original record. 9.c3) Transactions which execute a records disposition instruction whether for retention or destruction are documented by audit trails attached to the original record. 10. Removable: Records content and structure supporting the meaning of content must be deletable. 10a) Authority for deletion of record content and structure exists. 10b) Deletion transactions are documented as audit trails. 10c) Deletion transactions remove the content and structural information of records without removing audit trails reflecting context. Records - Usable 11. Exportable: It must be possible to transmit records to other systems without loss of information. 11a) Exporting protocols should be reversible or the lost functionality should be represented in a fashion that produces the same result in the target system as in the riginating environment. 12. Accessible: It must be possible to output record content, structure and context. 12.a Available: Records must be retrievable. 12.a1) The system must be able to retrieve the record of any transaction at any later date. 12.b Renderable: Records must display, print or be abstractly represented as they originally appeared at the time of creation and initial receipt. 12.b1) The structure of data in a record must appear to subsequent users as it appeared to the recipient of the record in the original transaction or a human meaningful representation of that original rendering should accompany the presentation of the original content. 12.c Evidential: Records must reflect the context of their creation and use. 12.c1) A human meaningful representation of the contextual audit trail of a record must accompany all displays or printed output. 13. Redactable: Records must be masked when it is necessary to deliver censored copies and the version as released must be documented in a linked transaction. 13a) The release of redacted versions of a record is a discrete business transaction. 13b) The fact of the release of a redacted version of a record is an auditable use of the original record and therefore results in creation of an audit trail with a link to the transaction which released the redaction. APPENDIX B: REFERENCE MODEL FOR BUSINESS ACCEPTABLE COMMUNICATIONS REGISTRATION METADATA RECORD DECLARATION Record-Declaration [Mandatory] IDENTIFIER Transaction-Identifier [Mandatory] Domain-Identifier [Mandatory] TERMS & CONDITIONS METADATA RIGHTS STATUS METADATA Access-Rights-Status [Mandatory; provided by the access resolver at time of record creation and meaningful to the resolver] Use-Rights-Status [Mandatory; provided by the use resolver at time of record creation and meaningful to the resolver] ACCESS METADATA Access-Basis [Mandatory] Payment-Terms-Resolver-Address [Required for Restricted Data]
Permission-Terms-Resolver-Address [Required for Restricted Data]
Identity-Terms-Resolver-Address [Required for Restricted Data]
Time Rule-Resolver-Address [Required for Restricted Data]
USE METADATA Use-Basis [Mandatory] Use-Citation [Mandatory] View Only-Copy Protection-Key [Optional] License-Terms-Resolver-Address [Required for Licensed Data]
RETENTION METADATA Retention-Period-End-Time [Optional] Disposition-Instruction-Code [Optional] Retention-Policy-Citation [Optional; good practice] Retention-Law/Regulation-Citation [Optional; good practice] Retention-Law/Regulation-Authority [Optional; good practice] Removal-Authority [Mandatory] STRUCTURAL METADATA FILE IDENTIFICATION File-Id [Mandatory] File-Authentication-Key [Mandatory] File-Modality [Mandatory] File-Encoding-Base [Mandatory, if not digital base 2] File-Data-Encoding-Type [Mandatory] Compression-Method [Mandatory] Encryption-Key:Address [Mandatory for Encrypted data] FILE RENDERING METADATA Application-Dependency [Mandatory] Software-Environment-Dependency [Mandatory] Hardware-Dependency [Mandatory] Rendering-Rules [Mandatory; Modality specific/domain specific] Dimensions [Mandatory; Modality specific/domain specific] Metrics [Mandatory; Modality specific/domain specific] Representation-Standard/De Facto Standard [Mandatory] Representation-Standard-Version: [Mandatory] RECORD RENDERING METADATA File-Linking-Rule/Standard: [Mandatory] File-Interchange-Standard:Version [Mandatory] CONTENT STRUCTURE METADATA Content-Structure [Mandatory] Content-Data Set [Mandatory] Application-Dictionary [Mandatory if no other content data set] Delimiters/Labels [Mandatory] Data Value-Lookup Tables [Mandatory, where existing] Data View-at Creation [Mandatory, if partial view] Version-Relationships [Mandatory, if prior version exists] Set-Relationships [Mandatory, if other set members exist] Hierarchical-Relationships [Mandatory, if higher/lower exist] Domain-Description-Standard [Optional] Domain-Access-Terminology [Optional] SOURCE METADATA Data Source-Type [Mandatory] System-Data Source-Documentation [Mandatory, if system is source] Data Capture-Instrument-Type [Mandatory, if instrument] Data Capture-Instrument-Documentation [Mandatory; Domain Specific] Source Data-Natural Language [Optional] Source Data-Quality [Optional, Domain Specific] Source Data-Protection-Flags [Mandatory] CONTEXTUAL METADATA TRANSACTION CONTEXT Originator-Identification [Mandatory] Recipient-Identification [Mandatory] Copy-Identifier [Mandatory] Business-Transaction Procedure Reference [Optional] Linked-Prior Event [Mandatory] Action-Requested [Mandatory if contractual obligation] User Specific-Configuration Data [Optional, good practice] RESPONSIBILITY Originating-Organization [Mandatory] Organization Structure-Process Link [Mandatory] Authorization [Mandatory] Authorization-Resolver-Address [Optional] BUSINESS FUNCTION Domain-Declaration [Optional] Domain Relevant-Accuracy [Optional, Domain Specific] Domain Relevant-Completeness [Optional, Domain Specific] Domain Relevant-Quality [Optional, Domain Specific] Domain Relevant-Reference Data [Optional, Domain Specific] Domain Relevant-Genre/Form [Optional, Domain Specific] CONTENT CONTENT-DESCRIPTION Descriptor-Standard [Optional, Domain Specific] Descriptor-Types [Optional, Domain Specific] Descriptors [Optional] RECORD Content-Created [Optional*] Content-Incorporated [Optional*] USE HISTORY Use-Type [Mandatory] Use-Value [Mandatory for file, index, classify, dispose etc.] Use-Instance-Time [Mandatory] Use-Instance-User [Mandatory] Use-Redactions [Mandatory if redacted on release]