Section 15 – Data Exchange

Data Exchange

Introduction

Animal Recording involves the collection, storage and exchange of data. Over time the use of electronic systems has spread from initially, in the early 1970s, being used just for centralised data processing to now (in 2016) being used in all aspects of animal recording.  The electronic systems for collecting, storing, transmitting, and processing data and information have evolved to be used widely by technicians, farmers, advisors and central systems.  The evolution is continuing at a rapid pace worldwide.  With this spread of electronic systems the need for standards to facilitate the ready transfer of data between systems has also grown very rapidly.

This part of the ICAR guidelines (chapter 15) is devoted to standards for facilitating data transfers and to processes for ensuring electronic systems comply with the ICAR guidelines.

Goals

The goals of these guidelines are:

1. To provide the developers of data processing systems with standards and processes that they can use to ensure the devices and systems they develop can be readily and reliably linked with the full range of animal data information systems used by ICAR members.
2. Harmonization of the definitions of exchanged data in order to be able to exchange information between heterogeneous information systems.
3. Development of global consensual data dictionaries for livestock.
4. Development and installation of standardized systems to support data exchange between information systems and farm equipment.

Scope

Animal data exchange consists of:

1. Business requirements and technical specifications.
2. Data descriptions provided by files in compliance with the recommendations from W3C for XML schemas, UNCEFACT and ISO.
3. Interface specifications provided by files in compliance with the recommendations from W3C for web service description languages (WSD).

This Guideline gives business requirements which are divided into three types:

1. General business requirements which address any type of data exchange.
2. Specific business requirements which address a given type of data exchange (Procedure two to five).
3. Data descriptions (Appendix A).

For a given type of data exchange, specific business requirements detail the:

1. purpose of data exchange;
2. business context of data exchange; and
3. requirements to exchange data which includes exchanged message descriptions.

Data description encompasses:

1. Service description
2. Message composition
3. Entity description
4. Data item description
5. Code set description

Technical implementation deals with the implementation of the same business requirements according different ways:

1. Primarily by using W3C and UNCEFACT recommendations for data and interface definition (SOAP).
2. Alternatively, by using the existing ISO standards (17532:2007) for data exchange on farm with stationery equipment and REST technology (This has to be elaborated more in detail in a future version of the ICAR ADE specification).

Presentation of animal data exchange

Background

At the moment, data exchanges between equipment and external information systems do not exist or require a sort of middleware (see Figure 1) between the equipment and the information system. It is based mainly on regional, manufacturer specific or outdated international standards (ISO ADED 1996 - ISO 11788-1, ISO 11788-2 , ISO 11788-3 see here for details).

Figure 1. Current situation for data exchange.

This middleware is costly to implement, difficult to maintain. It requires manual operations and does not allow to reduce the delay between an event and its registration by an external information system and to exchange large amount of data.

Equipped with modern high-speed IT infrastructure todays animal recording data centres are capable of processing huge amount of data created by an exponentially growing number of on farm equipment, sensors and analytical methods. In order to get reliable access to this data a high level of automation and standardization of data exchange has to be organized.

A data exchange in real time of a large amount of data would allow:

1. animal breeding on new traits;
2. improved animal monitoring by the aggregation of data from different sources: equipment, analysis laboratories etc; and  
3. improved farm equipment calibration in order to get more accurate measurements.

Objective of animal data Exchange

The objective is to establish direct, permanent, reliable, easy to implement and to maintain, and cost-effective exchanges of large amounts of data both ways: between equipment and external information systems and from equipment to external information systems.

How to achieve the objectives

Content of ADE

This objective may be achieved by a framework consisting of:

1. an architecture to support data exchange;
2. standards for messages and data;
3. tools to facilitate its implementation; and
4. a reactive maintenance process.

The architecture is based on a service-oriented approach where the equipment is the client of the external information system which is a service provider (see Figure 2). This architecture may be implemented by using different information technologies which are specified by this document.

Figure 2. Animal data exchange architecture.

The standards encompass:

1. Business requirements to use the service
2. Business requirements to deliver the service
3. Business requirements for exchanged data
4. Semantic definition of exchange data as well as their code set
5. Syntax specifications for the exchanged data according W3C, UNCEFACT and ISO
6. Interface specifications according W3C, UNCEFACT and ISO

The tools to facilitate the implementation encompass:

1. A web site to download:
   • Files of xml data types (XSD files)
   • Files of web service specifications (WSDL files)
2. A test platform

ADE implementation

Figure 3 below describes the process.

Figure 3. Life cycle of the framework.

The process is triggered by a business need.

Business requirements should be the result of collaboration between ICAR and manufacturers. They are independent from the techniques used to implement them.

Two types of implementations are considered:

1. W3C and UNCEFACT standards, the primary one; and
2. ISO (17532:2007) / REST, the upcoming alternatives.

The W3C implementation consists of:

1. Data element technical description as “xsd” files based on UNCEFACT data types.
2. Interface technical specifications as “wsdl” files.
3. From WSDL:
   • Client implementation by the manufacturer
   • Server implementation by the external information system

Business context

Figure 4 describes the business context.

Figure 4. Background business context.

Service providers are organizations who provide milking equipment owners with services whose purpose is:

1. to have their data registered by a multipurpose data base, and
2. to deliver aggregated information.

These organizations may be milk recording organizations, breeding organizations or other data base service providers.

The owner of the milking equipment and the service provider should agree on the conditions to use services and namely the parameters which are required for data exchange. Generally, this agreement is formalized by a contract.

Milking equipment is a set of hardware and software encompassing at least the following functions:

1. Measuring milk quantity and characteristics.
2. Filling bottles of milk sample and registering the links between the bottle identifier and that of the animals.
3. Storing milking results.
4. Connecting, sending and receiving data from external information systems.

Milking equipment may have several other functions which are not considered by that data exchange: animal monitoring, milking monitoring. However, characteristics not directly linked to milking but recorded during the milking process, e.g. weighing in the milking box, could be transported.  

A milking system may correspond to different types of equipment: robot, electronic milking meters etc.

The milking equipment is operated under the supervision of a milking operator who is staff dependent from the owner and designated as ‘Operator’ throughout the rest of the document. A milking operator has the right and the capacity:

1. to feed the milking equipment with parameters for data exchange and milk samples;
2. to install and to remove the bottles used for milk sample; and
3. to send milk samples to a milk analysis laboratory.

A milking equipment operator may correspond to different actors: farmer, a staff dependent from a farmer, a technician from a service provider such as a milk recording organizations etc.

The service provider operates an information system which provides the services. That information system consists of server, data bases and software, which are not a part the milking equipment and which is connected to the milking equipment by a network. It has at least the following functions:

1. Receiving and processing requests from the milking system.
2. Updating the data base of the information system according the requests of the milking system.
3. Providing the results of milk analytic laboratories which analyze the samples collected by the requesting milking equipment.

General Specifications

The general specifications are contained in Procedure 1 of these Section 15 ICAR Guidelines – link here.

Specific Services

Milking

Specifications for milking services are contained in Procedure 2 of these Section 15 ICAR Guidelines – link here.

Exchange of Animal Data

Specifications for exchange of animal data services are contained in Procedure 3 of these Section 15 ICAR Guidelines – link here.

Exchange of Reproduction Events

Specifications for exchange of animal data services are contained in Procedure 4 of these Section 15 ICAR Guidelines – link here.

Technical Services

Specifications for exchange of technical services are contained in Procedure 5 of these Section 15 ICAR Guidelines – link here.

Data Dictionary

The data dictionary is contained in Appendix A to these Section 15 ICAR Guidelines – link here.

References

1.      Semantics for Smart Dairy Farming: a milk production registration standard – SDF June 2013

2.     UN / UNCEFACT Modeling Methodology User Guide (CEFACT / TMG/N093)

3.     UN / UNCEFACT Business Requirements Specifications Document Template (CEFECT/ICG/005)

4.     ISO 11787: Electronic data interchange between information systems in agriculture —Agricultural data interchange syntax

5.     ISO 11788: Electronic data interchange between information systems in agriculture — Agricultural data element dictionary —Part 1: General description —Part 2: Dairy farming

6.     ISO 17532: Stationary equipment for agriculture —Data communications network for livestock farming

7.      ISO 11784: Radio frequency identification of animals - code structure

8.     ISO 3166 -1: Country code

9.     ICAR Guidelines

Data Exchange

Procedure 1: Methodology

Procedure 2: Milking data

Procedure 3: Animal data

Procedure 4: Reproduction data

Procedure 5: Technical data

Appendix A: Data dictionary