Electronic Data Interchange: An Overview of EDI Standards for Libraries (1993)

2. EDI STANDARDS

This chapter will describe the two main EDI standards that are currently used in North America and Europe; the ASC X12 group of standards supported by the American National Standards Institute (ANSI) and the EDIFACT standards supported by the United Nations Economic Commission for Europe (UN/ECE).

The authors acknowledge that this chapter includes a great deal of terminology and reaches a level of detail that may not be of interest to all readers. It is included as a primer for those interested in learning more about the specifics of EDI standards and their components.

2.1 Components of EDI Standards

1. A syntax and encoding scheme for messages which specifies the structure of data. The data should be independent of systems, machine and media constraints and should allow for human interpretation of the data transferred. As well, the data elements or groupings which are part of standard messages should be independent of each other so that one part may be changed without affecting any other part.

2. A data dictionary. This component of EDI standards defines the standard business data elements, such as date, time, delivery address, and currency used to create messages.

3. Combinations of data elements to be used for standard messages. A paper invoice, for instance, normally consists of a header portion stating the name and address of the billing party, the name and address of the paying party, the date of the invoice, an account number, etc. There is then a detail portion which consists of a series of invoice lines, each giving details of a billed transaction such as date, order number, number of units, item number, item description, unit price, and total price. There may also be a summary portion which gives totals. Each of these sections has an equivalent in EDI format with data elements combined into "segments" and segments combined into "messages".

2.2 ANSI X12

2.2.1 American National Standards Institute

ANSI itself does not develop standards but provides a forum for all concerned interests to identify standards needs, plan to meet these needs and agree on standards. ANSI is also the U.S. member of non-treaty international standards organizations such as the International Organization for Standardization (ISO) and the International Electrotechnical Committee (IEC). As such, ANSI coordinates U.S. participation in these groups.

2.2.2 Accredited Standards Committee (ASC X12)

The Data Interchange Standards Association (DISA) serves as the secretariat for ASC X12. DISA's activities include communicating with ANSI and the public on behalf of the ASC X12 committee, organizing X12 meetings and publishing the X12 standards. To obtain ASC X12 standards or further information on ASC X12 activities contact:

Once a year, DISA publishes the entire set of X12 standards in a single volume, called a release. This includes revisions of previously published standards, as well as new draft standards approved by ASC X12 during the year (referred to as Draft Standards for Trial Use). Releases are not considered American National Standards until they have undergone the ANSI-required public review process. ASC X12 uses the releases in order to make the approved draft standard available to users on a more frequent schedule. The Draft Standards for Trial Use are reviewed by ANSI every three years. Once approved they are adopted as American National Standards.

2.2.3 The X12 Series of Standards

• Data element dictionary (X12.3)
  
• Interchange control structure (X12.5)
  
• Application control structure (X12.6)
  
• Data Segment Directory (X12.22)
  
• Functional Acknowledgment (X.20)

2.2.4 X12 Syntax and Message Structure

The ASC X12 standards specify:

• the segments used in a transaction set,
  
• the sequence in which the segments must appear
  
• whether segments are mandatory or optional,
  
• when segments can be repeated
  
• how loops are structured and used

X12 message formats consist of the following components (refer to Figure 2.1).

Interchange Control Structures
Functional Groups
Transaction Sets
Segments
Data elements

Interchange Control Segments

These segments constitute the electronic structure which surrounds the transaction sets to be transmitted. They signal the beginning and end of organizational units of information within the message but do not contain data relevant to the EDI transaction. They indicate the sender of the message, the intended recipient, the date and time of transmission and the version of X12 in use.
The other type of control segment is associated with loops of segments that may be repeated within a message. A loop would be used to indicate the names and addresses of various parties in a purchase order transaction, if the billing address is different from the shipping address for a particular order. The loop header and loop trailer segments indicate the start of the loop and the end of the loop (Bass, 1991).

Figure 2.1 - Components of an X12 Transaction (58K)

Functional Groups

A functional group is a group of similar transaction sets (eg. three purchase orders). For example, an X12 interchange could consist of a functional group of purchase orders and a functional group of payment advices (refer to Figure 2.1). The receipt of an X12 interchange, functional group and transaction set must be acknowledged by means of functional acknowledgment. This indicates to the sender whether or not the interchange was received and whether the syntax can be processed.

Transaction Set

A business transaction is defined by a transaction set composed of a number of segments of variable length. Each segment is in turn composed of a number of data elements of variable lengths. A transaction set is analogous to a business document such as a purchase order, while a segment is analogous to a line of information in that purchase order and a data element is analogous to unit of information in the item line. For example, in the purchase order for a book, the number of copies requested or the unit price would be represented by data elements.

Each transaction set starts with a transaction set header (ST), followed by a "beginning segment" that uniquely identifies the type of transaction set. The transaction set header contains the transaction set identification and transaction set control number. This is followed by other segments (which may also be found in other transaction sets) and concluded by a transaction set trailer. The transaction set trailer (SE) is the last element in the transaction set. It defines the end of the transaction set and contains the number of segments included and the transaction set control number.

The transaction set identifier is a three-digit number by which the transaction set is known in both printed and electronic documentation. For example, the purchase order transaction set is transaction set 850. Each transaction set has three general areas:

1. Header area - contains segments with information common to the entire document

2. Detail or line item area - contains segments which provide detailed information such as quantity and price. The sequence of segments may form a loop which may be repeated up to a predefined maximum number of times. Each loop would roughly correspond to each item line of a paper invoice.

3. Summary area - contains summary information such as total price.

A transaction set specification indicates the segments used and the order in which they appear. A segment is specified to be included in a transaction set by citing its identifier as assigned in the Data Segment Directory. Some segments appear in every transaction set (e.g. PER, "Administrative Communications Contact", i.e. PERson to contact) while others are used in only a single transaction set.

The following lists the ASC X12 Transaction Sets that have been developed or are under development to support the book and serials industry (Santosuosso, 1992)

ASC X12         
Transaction Set          ASC X12                       Type of
Identifier Code          Description                   Data
_________________________________________________________________________________

810                      Invoice                       Invoices
832                      Price/Sales Catalog           Price Catalogs
850                      Purchase Order                Purchase Orders
855                      Purchase Order                Purchase Order
                         Acknowledgement               Acknowledgement
856                      Ship/Notice Manifest          Dispatch Information
860                      Purchase Order Change         Cancellations
869                      Order Status Inquiry          Claims
870                      Order Status Report           Claims Responses

Segments

A segment is an intermediate unit of information in a transaction set. It consists of logically related data elements in a defined sequence: a predetermined segment identifier (which is not a data element), one or more data elements, each preceded by a data element separator and followed by a segment terminator.

Segments available for use in X12 transaction sets are defined in the "Data Segment Directory" (ANSI X12.22), where each segment is represented as a diagram that specifies the segment identifier and the data elements it contains in their specified order.

Each segment has a unique identifier that consists of two or three alphanumerics. Within a transaction set, some segments may be specified as repeating, and some groups of segments may be repeated as loops.

Data elements

The data elements available for use within X12 segments are defined in the "Data Element Dictionary" (ANSI X12.3). Data elements have no explicit identifier, and are identified only by their sequence in the segment. A data element may be one of 6 types: numeric, decimal, identifier, string, date or time. All types are represented as character strings such as "AN" (alphanumeric) or "DT" (date). Data elements are also characterized by minimum and maximum lengths, and an indication of whether their presence is mandatory or optional.

The basic X12 character set consists of uppercase letters A-Z plus digits plus basic punctuation characters (!"#&'()*,-./:;?=). By special agreement, an extended character set may be used, which includes lowercase letters and additional punctuation characters.

X12 does not define any specific characters to be used as delimiters. Delimiters are instead defined for each interchange by their first use in the interchange start (ISA) segment. The asterisk (*) is the preferred data element separator, but is not required. NL (new line or carriage return) is preferred as the data segment terminator. The only constraint on the choice of delimiter characters is that they are not to be used elsewhere in an interchange.

Segments and data elements are intended for global use; that is, they are defined independently of any transaction sets in which they may be used. A segment specification indicates the data elements used and the order in which they appear. A data element specified to be included in a segment by citing its reference number in the data element dictionary (X12.3).

2.2.5 X12 and NISO Z39 standards

As NISO Z39 standards have been designed to store and archive bibliographic data, they focus on the content of the bibliographic item so it can be identified with certainty. The X12 formats on the other hand are more concerned with transmitting and receiving information and tend to contain less product description information.

In 1989, four members of the Book Industry Standards Advisory Committee (BISAC) agreed to migrate from the Z39.49 book order formats to the ANSI X12 format for purchase orders (BISAC Press Release, 1989). A number of other companies in the publishing and information industry followed suite. As publishers need to communicate with other retail sectors such as bookstores and grocery stores in order to do business, the ANSI X12 standards were more suited to their operations

In 1990, NISO adopted the X12 standards for the transmission and use of electronic information. BISAC and its sister organization SISAC moved in this direction as well (Cline McKay and Piazza, 1992). The adoption and development of these standards by book and serial industry groups is discussed in more detail in Chapter 5 (EDI Partners for Libraries).

2.3 EDIFACT

2.3.1 History and Origin

In 1971, SITPRO, the Simplification of Trade Procedures Board in Great Britain began work on common EDI standards for Europe. In 1974, the UK EDI syntax called Trade Data Interchange (TDI) was published and was first used by UK customs authorities. In 1975, the UN began to develop terms of reference for international EDI standardization. In 1979, the United National Guidelines for Trade Data Interchange (UN/GTDI) syntax, based on the TDI guidelines developed by SITPRO was published.

By this time, the ANSI X12 standards were in use in North America. The value of merging the two to develop an international EDI standard was recognized and work was initiated within the United Nations/Economic Commission for Europe (UN/ECE) to develop the international EDIFACT standards.

2.3.2 Organization of UN/EDIFACT

intergovernmental and non-governmental international organizations may participate in specific committees. Note that the UN/ECE should not be confused with the Commission of the European Community (CEC) (UN/EDIFACT Rapporteurs' Team (1990).

International EDI standardization activity is centered around the UN/EDIFACT standards developed by the United Nations Working Party on Facilitation of International Trade Procedures (WP.4). The structure of this group is illustrated in Figure 2.2. Work is carried out by a number of subordinate and related bodies including the Group of Experts on Data Elements and Automatic Data Interchange (GE.1) and the Group of Experts on Procedures and Documentation (GE.2). There are also a number of working groups preparing message standards for specific commercial sectors such as:

• Purchasing
  
• Finance
  
• Insurance
  
• Materials Management
  
• Travel, Tourism & Leisure
  
• Transportation
  
• Statistics

Working Party 4 meets twice each year. The link between WP.4 and the EDIFACT Boards where the actual standards development takes place is the Rapporteurs who report the results reached within the various EDIFACT Boards to WP.4. There are regional EDIFACT Rapporteurs and associated infrastructures for North America, Western Europe, Eastern Europe, Japan & Singapore, and Australia/New Zealand and an EDIFACT Board for Africa was recently established. A group has also been established to develop multilingual terminology that would make it possible to implement UN/EDIFACT in languages other than English. The group will also work to establish procedures and translation of the various working documents.

Figure 2.2 - United Nation EDIFACT Organizational Structure (188K)

United Nations Standard Messages (UNSMs)

When an agreement is reached on a new EDIFACT message, the EDIFACT Boards recommend the message for formal approval by the UN/ECE as a United Nations Standard Message (UNSM). Messages are classified according to their level of development.

Status 2 - stable, fully-approved United Nations Standard Message (UNSM)

Status 1 - approved for trial use

Status 0 - under development

New Message Request (NMR) - used to advise developers of message work in its formative stages

Messages at the Status 1 stage can be implemented in an operational environment. This provides an opportunity to test a message extensively before its reaches the final stage. As of the March 1993, the number of messages at the various levels are (External Affairs and International Affairs Canada, 1993):

Status 2 - 27

Status 1 - 25

Status 0 - 113

NMR - 119

The EDIFACT effort has enjoyed widespread support and commitment from UN member states. Liaison with ISO is provided through ISO Technical Committee 154, "Documents and data elements in administration, commerce and industry". EDIFACT standards have been progressed through the ISO machinery using a "fast-track" procedure, whereby ISO members or liaisons can submit existing standards directly for ballot as an ISO draft international standard.

A number of intergovernmental organizations such as Commission of the European Communities (CEC), European Free Trade Association (EFTA) participate in the EDIFACT work. A number of UN specialized agencies including: General Agreement on Tariffs and Trade (GATT), and International Telecommunications Union (ITU), also contribute to EDIFACT efforts.

2.3.3 EDIFACT Message Structure and Syntax

Each EDIFACT transmission is defined as an "interchange" and is composed of a number of "segments". Each of these segments has a "tag" containing a code that uniquely identifies the segment. This tag may also contain an indication of the structure of the segment. The segment may also be comprised of one or more data "elements".

Each tag is separated from the first data element by a delimiter character. The same delimiter character is used to separate subsequent data elements. It is referred as a segment tag and data element separator. A segment is terminated by another delimiter character called a "segment terminator". A data element may contain sub-elements which are separated from one another by a third delimiter character: a "sub-element separator".

The EDIFACT standard specifies two levels of the syntax, one of which uses a "least common denominator" character set consisting of upper-case letters, numbers and punctuation marks referred to as level A. The delimiters in Level A are graphic characters:

' (apostrophe = segment terminator),
+ (plus = segment tag and data element delimiter),
: (colon = sub-element delimiter), and
? (question mark = an escape character to allow the delimiters to be used within data elements with their normal meanings).

The level B character set includes upper and lower case letters, numbers and punctuation. The delimiters are non-printing control characters.

There are also a number of "service" segments that must be included in all EDIFACT transmission. These are:

Interchange Header: UNB
Message Header: UNH
Message Trailer: UNT
Interchange Trailer: UNZ.

Figure 2-3 illustrates the structure of an EDIFACT interchange. Between the message header and trailer, a message contains a predefined sequence of segments as specified in the Standard Messages Directory. Segments may be repeated and may be nested in a hierarchical manner to allow complex data structures to be transmitted.

The content of each segment is a predefined sequence of data elements as specified in the Standard Segments Directory. Some data elements are simple, in that they contain a single, atomic, value; other data elements are composite, in that they contain more than one value, or sub-element ("component data elements" in the terminology of ISO 9735).

The presence of segments within a message and of data elements within a segment is either mandatory or conditional and is specified in the appropriate directory. The content of the service segments is specified in ISO 9735.

The content of other segments, and the syntax and semantics of data elements, are taken from ISO 7372, the UN Trade Data Elements Directory (UNTDED). Because data elements are identified by their sequential position within the segment, empty or null data elements can not be omitted. In all cases where an empty data element is followed by another data element, the data element separator must be retained:

TTT+data1+data2+++data5'

In this example, TTT is the segment tag, and the '+' is used to separate the data elements. The presence of three separators together indicates that the third and fourth data elements are empty. If, however, the empty data element(s) are at the end of the segment, they may be omitted.

The designers of a message can specify that segments be repeated up to a specified number of times. Indication of repetition may be either explicit, by using a repetition sub-element in the tag, or implicit, with no specific indication of the repetition of the segment other than its multiple occurrence. Data elements within a segment may also be repeated up to the number of times specified in the segment directory. This repetition is always implicit. Similarly, nesting of segments may be implicit or explicit. Explicit nesting is indicated by multiple repeat-count sub-elements; the number of such sub-elements indicates the level of nesting.

Figure 2.3 - Structure of an EDIFACT Interchange (90K)

2.4 Relationship between X12 and EDIFACT

As the ANSI X12 standards were already in use when work began to develop the EDIFACT standards, the format and structure of X12 messages were a useful model. As a result, X12 and EDIFACT messages share common structural characteristics:

• character-based encoding, with multiple levels of support for various encoding standards, e.g. telex, 7-bit ASCII

• tagged and delimited data structures

• a global set of data segments and a segment directory to define them.

• a global set of data elements and a data element directory to define them.

• a message of a predefined type consisting of a specific sequence of segments.

• implicit identification of data elements in a segment by location.

• an "interchange" consisting of either "functional groups" each of which contains one or more messages or one or more messages by themselves.

However, given that the existing European standards (UN/TDI) were also considered when developing the EDIFACT formats and structures, there are some differences between the two standards (Woods, 1989):

• EDIFACT uses composite data elements

• looping and nesting procedures are different

• 6 data elements types are defined in ANSI X12 while only three are defined in EDIFACT

• there is no provision in EDIFACT for optional fields

• EDIFACT allows for two levels of syntax.

The differences between the two standards are considered to be minor by the X12/North American EDIFACT secretariat. There is very little difference between the two syntaxes in the overall design of the transaction set/message. There are however some differences in the way segments and data elements are structured so that a message developed using X12 syntax cannot be deciphered by a system based on the EDIFACT syntax. Work is currently underway to align the two syntaxes so that software using the UN/EDIFACT formats will also be able to communicate with X12 based systems (Malhotra, 1991). Another solution is to develop a common data dictionary that could be used by software developers and system integrators (Olson, 1992). This approach would result in EDI software that supports both the EDIFACT and X12 data formats.