Section 15 – Data Exchange: Difference between revisions

From ICAR Wiki
No edit summary
No edit summary
Line 196: Line 196:
</div>
</div>


Procedure 1:
= Procedure 1: Methodology =


:[https://www.icar.org/Guidelines/15-Procedure-1-Methodology.pdf Procedure 1: Methodology]
== Introduction ==
:[https://www.icar.org/Guidelines/15-Procedure-2-Milking-Data.pdf Procedure 2: Milking data]
For electronic data exchange a lot of technologies exist and new technologies are continously evolving. However, in order to establish standardized data exchange we have to concentrate on a defined set of accepted and mature technologies and describe the usage in detail. 
:[https://www.icar.org/Guidelines/15-Procedure-3-Animal-Data.pdf Procedure 3: Animal data]
 
:[https://www.icar.org/Guidelines/15-Procedure-4-Reproduction-Data.pdf Procedure 4: Reproduction data]
This procedure, as part of Section 15 Guidelines, introduces  the common methodology and technology used by the specific business processes described in procedures 2, 3 and 4:  
:[https://www.icar.org/Guidelines/15-Procedure-5-Technical-Data.pdf Procedure 5: Technical data]
 
:[https://www.icar.org/Guidelines/15-Appendix-A-Data-Dictionary.pdf Appendix A: Data dictionary]
# '''General Data Transport Specifications:''' describes the applied web service transport protocols and data protocols
# '''Access to the Service:''' describes some aspects to be taken into account when implementing and installing the services
# '''Communication work flow:''' covers the communication work flow aspects in a business process session like request, response and data processing.
# '''Dealing with Local Requirements:''' describes in detail how locally obligatory data elements which are not covered by the ICAR scope can be added to the interfaces
# '''Common Components:''' describes all the common protocol components which are reused in the specific business processes
 
== Definitions and Terminology ==
Table 1 contains a list of important definitions for terms and abbreviations used in these guidlelines.
{| class="wikitable"
| colspan="2" |'''''Table 1.  Definitions of Terms used in these guidelines.'''''
|-
|'''Term'''
|'''Definition'''
|-
|TCP/IP
|Transmission Control Protocol
|-
|HTTP
|Hyper Text Transfer Protocol
|-
|XML
|Extensible Markup Language
|-
|XSD
|XML Schema Definition
|-
|SOAP
|Simple Object Access Protocol
|-
|WSDL
|Web Service Description Language
|-
|REST
|Representational State Transfer
|-
|ADIS
|Agricultural Data Interchange  Syntax
|-
|ADED
|Agricultural Data Element Dictionary
|-
|JSON
|JavaScript Object Notation
|-
|UNCEFACT
|UN/CEFACT is the United Nations  Centre for Trade Facilitation and Electronic Business
|-
|ISO
|International Organization for  Standardization
|-
|IANA
|Internet Assigned Numbers  Authority
|}
 
== Scope ==
gives a pictorial summary of the main elements of this procedure. The numbers in this figure refer to the heading numbers of this procedure.
[[File:Imagescope1.png|center|thumb|422x422px|''Figure 1. Scope of procedure 1 of Section 15 of ICAR Guidelines.'']]
 
== General Data Transport Specifications ==
 
=== SOAP/XML Web Services ===
For the primary data transmission method the W3C standard SOAP/XML version 1.2 has been chosen. Detailed descriptions of this standard can be found here:
 
http://www.w3.org/TR/2007/REC-soap12-part0-20070427/
 
Since a “top down” approach has been adapted, ICAR ADE provides WSDL files and a set of XSD files which make up a complete set of machine readable definition files from which SOAP web service implementations can be created. Most modern programming languages e.g. Java, .NET etc. provide tools which facilitate an easy setup process of interfaces classes to be used as a link between data transport layer and business process layer.
 
For each release of the ICAR ADE specifications a specific set of WSDL/XSD files can be downloaded from the ICAR ADE website – '''[https://icar-web.atlassian.net/wiki/spaces/ADE/overview link here]'''. See Table 2. ICAR ADE schema files” for a description of the files. The files will be provided in a zip archive file e.g. “ADE_Schema20150309_1.8.zip” for version 1.8 available '''[https://icar-web.atlassian.net/wiki/spaces/ADE/pages/1835044/WSDL-files+1.8 here]'''.
 
The WSDL/XSD files describe different aspects of the transport protocol:
 
# Service end points
# Input and output messages
# Basic data type definitions (XSD types, UNCEFACT types etc.)
# Complex data type definitions : (entity tags, item tags, compositions, attributes, usage of basic data types)
# Constraints (data type and value range restrictions)
 
{| class="wikitable"
| colspan="2" |'''''Table 2. ICAR ADE schema files'''''
|-
|'''File  Name /'''
 
'''File  Name Schema'''
|'''File Description'''
|-
|*.XSD
|XML Schema Definition files, machine readable  description of the ICAR ADE XML data structures
|-
|ICAR_*.XSD
|Data element definitions created by the ICAR ADE  group
|-
|ISO_*.XSD
|Data element definitions derived from ISO standards
|-
|UNECE_*.XSD
|XSD files provided by the UN/CEFACT project:
 
(see <nowiki>http://www.unece.org/cefact/xml_schemas/index.html</nowiki>)
|-
|UnqualifiedDataType_13p0.XSD
|XSD file which defines the unqualified UNCEFACT data  types. These data types extend the standard XSD data types with additional  attributes. In the ICAR and UNCEFACT definitions they are used instead of the  standard XSD types.
|-
|*_CODE_*
|XSD files defining code lists
|-
|wsMrAde.wsdl
|ICAR ADE SOAP service description, (WSDL = Web Service  Description Language)
 
It defines a set of message pairs each consisting in  a request and a response message.
|-
|ADE_v1p8.xsd
|This file provides the entry point to the ICAR ADE  message data structure definitions
|}
 
 
Throughout this document messages and data definitions are illustrated using graphical presentations of XSD structures created by the commercial software XMLSPY from ALTOVA.
 
See for example Figure 2 Example “GetHerdListRequest”
[[File:Mw.png|center|thumb|374x374px|''Figure 2 Example “GetHerdListRequest”'']]
 
 
Below the symbols used to represent the XSD schemas in the following pages:
 
Table 3 XSD schema symbols
[[File:Mw1.png|none|thumb|457x457px]]
 
=== Alternative Transport Protocols (ADIS/ADED/IsoAgriNet, REST) ===
Parallel to SOAP other international transport protocols exist. This is for example the ISO ADIS/ADED standard and its last extensions described in ISO 17532(2007).  On the other hand the REST protocol, based on the HTTP transport protocol using XML, JSON and other data encoding technology, has gained a broader acceptance in the domain of internet data transfer during the recent years.
 
The ADE-WG is working on the development of  a standardized  REST-API description.
 
=== ICAR Types and Code Lists ===
Elements created within the process of the ICAR ADE specification can be identified by the prefix “'''icar:'''” within the WSDL/XSD files and descriptions below.
 
All data items are based on UNCEFACT data types (see section UNCEFACT Data Types)
 
A detailed description of the data elements is given below.
 
=== External Elements ===
It is the aim of the ICAR ADE standardization process to make as far as possible use of elements already defined within existing standardization frameworks as ISO, IANA and UNCEFACT.
 
==== UNCEFACT Data types ====
The ICAR ADE WSDL/XSD definitions make extended use of UNCEFACT basic types.
 
UNCEFACT basic types are types derived from basic XSD types. The concept consists mainly of adding further attributes and restrictions to the XSD types. They are defined in file UnqualifiedDataType_13p0.xsd.
 
In the WSDL/XSD files and descriptions below those elements can be identified by the prefix “'''udt:'''”
 
Table 4 UNCEFACT basic data types” lists the UNCEFACT types currently used for the types of a data item.
{| class="wikitable"
| colspan="4" |'''''Table 4 UNCEFACT basic data types'''''
|-
|Name
|Description
|Comment
|XSD  Type
|-
|udt:CodeType
 
|A character  string that may be used to represent or replace the definitive value.
|A code is  referring to an enumeration
|String
|-
|udt:DateType
|A particular  point in the progression of date
|Representation  as defined by transport protocol
|String
|-
|udt:DateTimeType
|A particular  point in the progression of time
|Representation  as defined by transport protocol
|String
|-
|udt:IDType
|A character  string used uniquely to establish the identity of, and distinguish, one  instance of an object within an identification scheme from all other objects  within the same scheme.
|
|String
|-
|udt:MeasureType
|A numeric  value determined by measuring an object
|Measures are  specified with a unit attribute holding a measure unit, a resolution and when  it is required a minimum and a maximum value.
|Decimal
|-
|udt:NameType
|A word that  constitutes the distinctive designation of a person, place, thing, or concept
|
|String
|-
|udt:BinaryObjectType
|Base64 coded  binary data
|Used e.g. for  zip compressed message
|String
|-
|udt:TextType
|A general  text type
|
|String
|-
|udt:IndicatorType
|A Boolean  indicator or true and false
|
|Boolean
|}
 
==== UNCEFACT Code Lists ====
UNCEFACT provides an extensive set of code lists. See Table 5 UNCEFACT Code Lists” for the code lists used by the ICAR ADE specifications.
{| class="wikitable"
| colspan="2" |'''''Table 5 UNCEFACT Code Lists'''''
|-
|'''Code  List'''
|'''Description'''
|-
|AgencyIdentificationCode
|A list of agencies responsible for code list  maintenance to be used as value of the attribute '''listAgencyID''' in the UNCEFACT data type '''udt:CodeType'''
|-
|CharacterSetEncodingCode
|A list of character set encoding codes to be used as  value of the attribute '''encodingCode'''  in the UNCEFACT data type '''xsd:base64Binary'''
|-
|MeasurementUnitCommonCode
|An extensive set of commonly used units to be used  as value of the attribute '''unitCode'''  in the UNCEFACT data type '''udt:MeasureType'''
|}
 
==== ISO Code Lists ====
See Table 6 ISO code lists for the ISO code lists used by the ICAR ADE specifications.
{| class="wikitable"
| colspan="2" |'''''Table 6 ISO code lists'''''
|-
|'''Code  List'''
|'''Description'''
|-
|ISO3AlphaCurrencyCode
|A list of ISO currency codes to be used as value of  the attribute '''currencyID''' in the  UNCEFACT data type '''udt:AmountType'''
|-
|ISOTwoletterCountryCode
|A list of two letter ISO country codes to be used as  value of the item '''Country'''
|}
 
==== IANA Code Lists ====
See Table 7 IANA Code Lists for the IANA code lists used by the ICAR ADE specifications.
{| class="wikitable"
| colspan="2" |'''''Table 7 IANA Code Lists'''''
|-
|'''Code  List'''
|'''Description'''
|-
|CharacterSetCode
|A list of IANA character set codes to be used as  value of the attribute '''characterSetCode'''  in the UNCEFACT data type '''xsd:base64Binary'''
|-
|MIMEMediaType
|A list of IANA MIME media type codes to be used as  value of the attribute '''mimeCode''' in  the UNCEFACT data type '''xsd:base64Binary'''
|}
 
== Access to the Service ==
 
=== Exchange Parameters ===
The equipment should provide the operator with an interface to capture exchange parameters for a given service provider consisting of:
 
# Service provider URL
# Authentication information (user/password, token)
# Sender ID
# Sender name
# Sender country
# Recipient ID
# Recipient name
# Recipient country
# Type of location
# Type of identifier of the location
# Identifier of the location
# Name of the location
# Country of the location
# Type of primary animal identifier
# Type of secondary animal identifiers (e.g. on farm animal id, animal name)
# Other manufacturer or service provider specific parameters
 
For a given service provider the exchange parameters should be the same for all the consumed services.
 
=== Data Transmission ===
As the services are not permanently accessible, no request should be sent by the equipment before having checked if there is access to:
 
# The network
# The service provider server
# The service itself
 
 
Any new data should be registered by the information system as soon as possible.
 
No new data should be sent to the information system as long as previous data is still being processed. The service consumer has to wait for a processing result or a ticket in case of asynchronous processing.
 
No ticket should be sent to the information system before the time given by the information system.
 
Any message whose syntax is not in compliance with the syntax requirements should not be sent.
 
As soon as a data is included in a request an answered by a response it should be considered as sent to the information system.
 
Service providers and consumers should be prepared for updates and deletes. The service provider should take also take into account that the sending party is declaring updates as a create operation not knowing that the data is already present in the service provider’s data base. (e.g. in case of retransmission initiated by the consumer)
 
Preparing a request in compliance with business requirements should be performed separately from preparing a message whose syntax depends on the type of technical mapping (web service with SOAP at the moment).
 
=== Encryption ===
If possible encryption should be used in order to insure privacy and authenticity. The process of encryption itself is not described here. The encryption capabilities of the applied transport protocol should be used e.g. HTTPS for HTTP-Transport.
 
=== Compression ===
In order to minimize the transport delay of big messages, especially when coded in xml, compression should be applied. Some transport protocols offer compression features like transparent gzip compression of http responses. However since http compression is only applicable to the response it is not usable for the compression of data loads in http requests.
 
In order to avoid this limitation a special XML based compression mechanism is described for ICAR data exchange. Specific messages can be defined as zip compressible offering the alternative of uncompressed or zip compressed Base64 binary encoded xml structures both for request and response. For details see chapter “Message Design” below.
 
=== User Right Verification ===
The right of a given user to use a given service is verified by either using an “'''authentication token'''” which prevents him from sending a user ID and a password at each request or by a '''user name authenticated by a password'''.
 
'''''Authentication by'' user name and password ''should only be used with encrypted transport'''''.
 
== Communication work flow ==
 
=== Data Processing ===
Any service may be provided either in real time or in time differed.
 
A response should be sent to any response within several seconds.
 
Any request and any response should be stored by the equipment and the information system.
 
The requirements to update the data base of the information system differ from one service provider to another one.
 
Any request which is not in compliance with one the following conditions should not be processed:
 
# Compliance with the syntax
# Validity of the identification token
# Validity of the ticket if any
 
=== Request Specification ===
Any request should consist of three parts:
 
# A single '''message header'''  (see the entity MessageHeader of Type “ADEExchangedDocumentType”  in  ‘Data description’) The sender of the request creates and provides his own unique MessageID  (Item ADEExchangedDocumentType.Identifier). See chapter “Best Praxis for MessageID Creation” below.
# A single '''standard request''' (see the entity StandardRequest in ‘Data description’)
# A '''specific request''' which may consist of  a specific message for a given service  or the zip compressed base64 binary of the specific message or a ticket. A ticket is sent in order to retrieve the results of an asynchronous message processing task. (see diagrams below)
 
[[File:Mw2.png|center|thumb|''Figure 3 Request with a specific request'']]
 
 
=== Response Specification ===
Any messsage response should consist of three parts:
 
# A single message header (see the entity MessageHeader of Type  “ADEExchangedDocumentType” in ‘Data description’) The sender of the response creates and provides his own unique Message id. (See chapter “Best Praxis for MessageID Creation” below)
# A single standard response (see the entity StandardResponse in ‘Data description’. In the standard response the original MessageID from the sender is returned in the item RequestID. The standard response contains the item RequestProcessingStatus which identifies the status of the request processing on the side of the service provider:
 
{| class="wikitable"
| colspan="2" |'''''Table 8 RequestProcessingStatusCodeType (base xsd:string)'''''
|-
|'''Key'''
|'''Description'''
|-
|O
|Processed without errors
|-
|P
|Data accepted for  asynchronous processing (client can retrieve the processing result later  using provided ticket)
|-
|E
|Processed and  rejected due to errors
|-
|W
|Processed with warnings
|}
c. A specific response which may consist of either a ticket or specific message for a given service or the zip compressed base64 binary of the specific message.
 
The existence and content of the specific response depends on the definition of the service and the content of the RequestProcessingStatus item.
 
In case of RequestProcessingStatus = ‘P’ the specific response holds a Ticket
 
If a specific response message is defined it can either be represented uncompressed or zip compressed of type Base64binary.
 
If RequestProcessingStatus = ‘P’, ‘E’ or ‘W’ there might be no specific response at all depending on the message definition.
[[File:Mw3.png|center|thumb|''Figure 4 Response with a Specific Response'']]
[[File:Mw4.png|center|thumb|498x498px|''Figure 5 Specific Response Schema'']]
[[File:Mw5.png|center|thumb|''Figure 6 Standard Response'']]
 
 
=== Best Praxis for Message-Id Creation ===
The message ID created by both service consumer and service provider should be globally unique in order to allow a simple identification of each message e.g. for tracing and debugging purposes.
 
The MessageID should be based on a chain of unique hardware identifier, a unique software identifier and an additional distinguishing component large enough to uniquely identify messages created at very fast rates. This dynamic component could be a timestamp with ms precision or a sequence. The unique hardware identifier could be the globally unique MAC address of the network interface. The unique software identifier should be unique on the system running the software.
 
There are tools in different programming languages which provide simple means of unique id creation e.g. Guid.NewGuid() in C#
 
== Dealing with Local Requirements ==
The ICAR ADE standardization process cannot take into account all local requirements. In the first place it offers a common basis to build on.
 
In the first place “local” is a placeholder for national standards. However it is also possible to add local extensions based on an agreement between a service consumer and a service provider (e.g. for special projects, prototypes etc.).
 
In order to allow local extensions and to fill standardization gaps which cannot be covered by ICAR the ADE specifications provide two ways of local variations and extensions:
 
# '''Local Code Lists''': Local code lists for all items of type '''udt:CodeType''' for which ICAR is not yet able to provide a commonly accepted code set
# '''Local Additional Data''': An optional list of key value pairs in each specific entity to be used for locally required data elements
 
Local variation should be reduced to a minimum since it is contradictory to the goal of ICAR to establish global standards, leads to increased complexity, complicates software development and creates additional costs.
 
=== 2.1        Local Code Lists ===
Local code lists must be provided for items of type '''udt:CodeType''' in the context of a local standardization process, e.g. breed codes.
 
The content cannot be checked with the SOAP/XML validation procedure but must be evaluated in an extra program step before filling the XML data structures and after extracting the data from the XML structures. It is the responsibility of the software developers to assure that the data exchange is in accordance to ICAR and to local specifications.
 
In order to help manufacturers with the maintenance and integration of local code lists these specifications define the service '''GetLocalCodeList''' which can be used to query local code lists from the local service providers. For details see section “Technical Services”.
 
=== Local Additional Data ===
Sometimes it is necessary for local implementations of the ICAR messages to add local data items to the specific messages which are not known or not covered by the ICAR specification process.
 
For those scenarios a simple and flexible extension ('''LocalAdditionalData''') has been added to each specific entity.
 
It consists of an optional list of code/value pairs of type udt:textType which can be filled according to local definitions.
 
It is the responsibility of the local business process owners to document and assure the proper handling of this code/value pairs. The ICAR wsdl/xsd definitions only check the proper usage of the XML structure not the content.
 
For details see the wsdl/xsd files and the section
 
“CommonComponents”-”LocalAdditionalDataType”.
 
== Common Components ==
This part describes common components which are used in all the specific services described in anex B, C and D.
 
=== ADE Exchanged Document Type ===
The entity '''ADEExchangedDocumentType''' (Figure 7 ADEExchangedDocumentType) gives information describing general aspects of the message.
 
Used by data element: '''MessageHeader'''
[[File:Mw6.png|center|thumb|540x540px|''Figure 7 ADEExchangedDocumentType'']]
 
# '''Identifier''': Unique identifier of the message given by the sender
# '''Issueing''': The date and the time where the message is issued.
# '''Version''': Version number of the message
# '''Language''': Language used for the message
# '''LocalAdditionalData''': Optional list of key/value pairs used in a local context as described in section “Common Components – Local Adaptions”
# '''SenderParty''': Organization or person responsible for the content of the message, not a server identifier.
# '''RecipientParty''': Organization or person responsible for processing the message, not a server identifier.
 
=== ADE Party Type ===
The entity '''ADEPartyType''' (Figure 8 ADEPartyType) gives the name and the identifier of a party which may be either the sender or the recipient of a message.
 
Used by data elements: '''SenderParty''', '''RecipientParty'''
[[File:Mw7.png|center|thumb|493x493px|''Figure 8 ADEPartyType'']]
 
* '''Name''': Identifier of the specified party
* '''ID''': Identifier of the specified party
* '''Country''': Country of the specified party in accordance to code list '''ISOTwoletterCountryCodeIdentifierContent'''
 
=== Standard Request Type ===
The entity '''StandardRequestType''' (Figure 9 StandardRequestType) gives the standard content of a request.
 
Two different types of authentication can be used, token or user/password based.
 
Used by data element: '''StandardRequest'''
[[File:Mw8.png|center|thumb|534x534px|''Figure 9 StandardRequestType'']]
 
# '''AuthenticationToken:''' temporal code used for verification of the user’s identity and of its right to use a service.
# '''AuthentificationLogin''': Pair of username and password used for login
# '''UserName''': User name used for authentication
# '''Password''': Password used for authentication
 
=== Standard Response Type ===
The entity '''StandardResponseType ('''Figure 10 StandardResponseType''')''' gives the standard content of a response.
 
Used by data element: '''StandardResponse'''
[[File:Mw9.png|center|thumb|531x531px|''Figure 10 StandardResponseType'']]
 
# '''RequestProcessingStatus''': Status of request processing returned by server according to code list '''RequestProcessingStatusCode'''
# '''RequestID''': Copy of the message identification in the '''request’s MessageHeader.Identifier''' item
# '''RequestProcessingError''': In case an error happens during the processing of the request an error description is provided here
 
=== Error Type ===
The entity '''ErrorType''' (Figure 11 ErrorType) gives the request processing errors.
 
Used by data element: '''RequestProcessingError'''
[[File:Mw10.png|center|thumb|400x400px|''Figure 11 ErrorType'']]
 
# '''ErrorID''': Identifier the error, e.g. error text message, local error code etc.
# '''ErrorSeverity''': Severity of the error according to code list '''ErrorSeverityCode'''
# '''ErrorDescription''': Human readable description of the error as text
 
=== Ticket Response Type ===
The entity '''TicketResponseType''' (Figure 12TicketResponseType) conveys a ticket in order to retrieve a response processed in time differed (asynchronous processing mode).
 
It is used in '''SpecificRequest''' and '''SpecificResponse''' by data element '''TicketResponse'''.
[[File:Mw11.png|center|thumb|443x443px|''Figure 12TicketResponseType'']]
 
# '''TicketID''': Identify the response to retrieve
# '''NotBefore''': Date time of probable response availability
 
=== ZIP Message Type ===
'''ZipMessageType''' (Figure 13 ZipMessageType) is a wrapper type designed to transport the zip compressed version of the specific data part of a message. Contrary to http compression method which can only be used for the response part this method can be applied to request and response. It is defined as a choice for each specific request or response which transports data entities.
[[File:Mw12.png|center|thumb|405x405px|''Figure 13 ZipMessageType'']]
 
# '''ZIPMessage''': zip compressed data
 
=== Local Additional Data Type ===
The entity '''LocalAdditionalDataType''' (Figure 14 LocalAdditionalDataType) is a container for code value pairs for local usage outside the ICAR specification scope.
 
It is used as an optional list element '''LocalAdditionalData''' in each specific entity.
[[File:Mw13.png|center|thumb|418x418px|''Figure 14 LocalAdditionalDataType'']]
 
# '''AdditionalDataCode''': Name of the code value pair
# '''AdditionalDataValue''': Value of the code value pair
# '''AdditionalDataComment''': Optional description of the local data
 
=== Time Period Type ===
The utility type '''TimePeriodType''' (Figure 15TimePeriodType) is a wrapper type designed to transport the beginning and end of a time period.
[[File:Mw14.png|center|thumb|422x422px|''Figure 15TimePeriodType'']]
 
# '''StartTime''': The start of the time period
# '''EndTime''': The optional end of the time period
 
== Naming Rules ==
Names of data elements, operations and messages use upper camel case.
 
Service name consist of:
 
# A prefix which gives the type of operation according the following:
#* Create: Insert data into a data base
#* Update: update of a data base
#* Get: retrieve data from data base
#* Delete: delete data from data base
# The name of the service For example UpdateMilkingResult is the service which allows updating a data base from milking results exchange. Message name consists of:
## The name of the service
## A suffix, gives the type of message:
##* Request
##* Response
 
For example, UpdateMilkingResultRequest is the request to trigger the service UpdateMilkingResult.
 
== References ==
1.       Semantics for Smart Dairy Farming: a milk production registration standard – SDF June 2013
 
2.      UN / UNCEFACT Modeling Methodology User Guide (CEFACT / TMG/N093)
 
3.      UN / UNCEFACT Business Requirements Specifications Document Template (CEFECT/ICG/005)
 
4.      ISO 11787 : Electronic data interchange between information systems in agriculture —Agricultural data interchange syntax
 
5.      ISO 11788 : Electronic data interchange between information systems in agriculture — Agricultural data element dictionary —Part 1: General description —Part 2: Dairy farming
 
6.      ISO 17532 : Stationary equipment for agriculture —Data communications network for livestock farming
 
7.       ISO 11784: Radio frequency identification of animals - code structure
 
8.      ISO 3166 -1 : Country code
 
9.      ICAR G

Revision as of 03:24, 6 September 2024

Data Exchange

Introduction

Animal Recording involves the collection, storage and exchange of data. Over time the use of electronic systems has spread from initially, in the early 1970s, being used just for centralised data processing to now (in 2016) being used in all aspects of animal recording.  The electronic systems for collecting, storing, transmitting, and processing data and information have evolved to be used widely by technicians, farmers, advisors and central systems.  The evolution is continuing at a rapid pace worldwide.  With this spread of electronic systems the need for standards to facilitate the ready transfer of data between systems has also grown very rapidly.

This part of the ICAR guidelines (chapter 15) is devoted to standards for facilitating data transfers and to processes for ensuring electronic systems comply with the ICAR guidelines.

Goals

The goals of these guidelines are:

  1. To provide the developers of data processing systems with standards and processes that they can use to ensure the devices and systems they develop can be readily and reliably linked with the full range of animal data information systems used by ICAR members.
  2. Harmonization of the definitions of exchanged data in order to be able to exchange information between heterogeneous information systems.
  3. Development of global consensual data dictionaries for livestock.
  4. Development and installation of standardized systems to support data exchange between information systems and farm equipment.

Scope

Animal data exchange consists of:

  1. Business requirements and technical specifications.
  2. Data descriptions provided by files in compliance with the recommendations from W3C for XML schemas, UNCEFACT and ISO.
  3. Interface specifications provided by files in compliance with the recommendations from W3C for web service description languages (WSD).

This Guideline gives business requirements which are divided into three types:

  1. General business requirements which address any type of data exchange.
  2. Specific business requirements which address a given type of data exchange (Procedure two to five).
  3. Data descriptions (Appendix A).

For a given type of data exchange, specific business requirements detail the:

  1. purpose of data exchange;
  2. business context of data exchange; and
  3. requirements to exchange data which includes exchanged message descriptions.

Data description encompasses:

  1. Service description
  2. Message composition
  3. Entity description
  4. Data item description
  5. Code set description

Technical implementation deals with the implementation of the same business requirements according different ways:

  1. Primarily by using W3C and UNCEFACT recommendations for data and interface definition (SOAP).
  2. Alternatively, by using the existing ISO standards (17532:2007) for data exchange on farm with stationery equipment and REST technology (This has to be elaborated more in detail in a future version of the ICAR ADE specification).

Presentation of animal data exchange

Background

At the moment, data exchanges between equipment and external information systems do not exist or require a sort of middleware (see Figure 1) between the equipment and the information system. It is based mainly on regional, manufacturer specific or outdated international standards (ISO ADED 1996 - ISO 11788-1, ISO 11788-2 , ISO 11788-3 see here for details).

Figure 1. Current situation for data exchange.


This middleware is costly to implement, difficult to maintain. It requires manual operations and does not allow to reduce the delay between an event and its registration by an external information system and to exchange large amount of data.

Equipped with modern high-speed IT infrastructure todays animal recording data centres are capable of processing huge amount of data created by an exponentially growing number of on farm equipment, sensors and analytical methods. In order to get reliable access to this data a high level of automation and standardization of data exchange has to be organized.

A data exchange in real time of a large amount of data would allow:

  1. animal breeding on new traits;
  2. improved animal monitoring by the aggregation of data from different sources: equipment, analysis laboratories etc; and  
  3. improved farm equipment calibration in order to get more accurate measurements.

Objective of animal data Exchange

The objective is to establish direct, permanent, reliable, easy to implement and to maintain, and cost-effective exchanges of large amounts of data both ways: between equipment and external information systems and from equipment to external information systems.

How to achieve the objectives

Content of ADE

This objective may be achieved by a framework consisting of:

  1. an architecture to support data exchange;
  2. standards for messages and data;
  3. tools to facilitate its implementation; and
  4. a reactive maintenance process.

The architecture is based on a service-oriented approach where the equipment is the client of the external information system which is a service provider (see Figure 2). This architecture may be implemented by using different information technologies which are specified by this document.

Figure 2. Animal data exchange architecture.
The standards encompass:
  1. Business requirements to use the service
  2. Business requirements to deliver the service
  3. Business requirements for exchanged data
  4. Semantic definition of exchange data as well as their code set
  5. Syntax specifications for the exchanged data according W3C, UNCEFACT and ISO
  6. Interface specifications according W3C, UNCEFACT and ISO

The tools to facilitate the implementation encompass:

  1. A web site to download:
    • Files of xml data types (XSD files)
    • Files of web service specifications (WSDL files)
  2. A test platform

ADE implementation

Figure 3 below describes the process.

Figure 3. Life cycle of the framework.


The process is triggered by a business need.

Business requirements should be the result of collaboration between ICAR and manufacturers. They are independent from the techniques used to implement them.

Two types of implementations are considered:

  1. W3C and UNCEFACT standards, the primary one; and
  2. ISO (17532:2007) / REST, the upcoming alternatives.

The W3C implementation consists of:

  1. Data element technical description as “xsd” files based on UNCEFACT data types.
  2. Interface technical specifications as “wsdl” files.
  3. From WSDL:
    • Client implementation by the manufacturer
    • Server implementation by the external information system

Business context

Figure 4 describes the business context.

Figure 4. Background business context.

Service providers are organizations who provide milking equipment owners with services whose purpose is:

  1. to have their data registered by a multipurpose data base, and
  2. to deliver aggregated information.

These organizations may be milk recording organizations, breeding organizations or other data base service providers.

The owner of the milking equipment and the service provider should agree on the conditions to use services and namely the parameters which are required for data exchange. Generally, this agreement is formalized by a contract.

Milking equipment is a set of hardware and software encompassing at least the following functions:

  1. Measuring milk quantity and characteristics.
  2. Filling bottles of milk sample and registering the links between the bottle identifier and that of the animals.
  3. Storing milking results.
  4. Connecting, sending and receiving data from external information systems.

Milking equipment may have several other functions which are not considered by that data exchange: animal monitoring, milking monitoring. However, characteristics not directly linked to milking but recorded during the milking process, e.g. weighing in the milking box, could be transported.  

A milking system may correspond to different types of equipment: robot, electronic milking meters etc.

The milking equipment is operated under the supervision of a milking operator who is staff dependent from the owner and designated as ‘Operator’ throughout the rest of the document. A milking operator has the right and the capacity:

  1. to feed the milking equipment with parameters for data exchange and milk samples;
  2. to install and to remove the bottles used for milk sample; and
  3. to send milk samples to a milk analysis laboratory.

A milking equipment operator may correspond to different actors: farmer, a staff dependent from a farmer, a technician from a service provider such as a milk recording organizations etc.

The service provider operates an information system which provides the services. That information system consists of server, data bases and software, which are not a part the milking equipment and which is connected to the milking equipment by a network. It has at least the following functions:

  1. Receiving and processing requests from the milking system.
  2. Updating the data base of the information system according the requests of the milking system.
  3. Providing the results of milk analytic laboratories which analyze the samples collected by the requesting milking equipment.

General Specifications

The general specifications are contained in Procedure 1 of these Section 15 ICAR Guidelines – link here.

Specific Services

Milking

Specifications for milking services are contained in Procedure 2 of these Section 15 ICAR Guidelines – link here.

Exchange of Animal Data

Specifications for exchange of animal data services are contained in Procedure 3 of these Section 15 ICAR Guidelines – link here.

Exchange of Reproduction Events

Specifications for exchange of animal data services are contained in Procedure 4 of these Section 15 ICAR Guidelines – link here.

Technical Services

Specifications for exchange of technical services are contained in Procedure 5 of these Section 15 ICAR Guidelines – link here.

Data Dictionary

The data dictionary is contained in Appendix A to these Section 15 ICAR Guidelines – link here.

References

1.      Semantics for Smart Dairy Farming: a milk production registration standard – SDF June 2013

2.     UN / UNCEFACT Modeling Methodology User Guide (CEFACT / TMG/N093)

3.     UN / UNCEFACT Business Requirements Specifications Document Template (CEFECT/ICG/005)

4.     ISO 11787: Electronic data interchange between information systems in agriculture —Agricultural data interchange syntax

5.     ISO 11788: Electronic data interchange between information systems in agriculture — Agricultural data element dictionary —Part 1: General description —Part 2: Dairy farming

6.     ISO 17532: Stationary equipment for agriculture —Data communications network for livestock farming

7.      ISO 11784: Radio frequency identification of animals - code structure

8.     ISO 3166 -1: Country code

9.     ICAR Guidelines

Procedure 1: Methodology

Introduction

For electronic data exchange a lot of technologies exist and new technologies are continously evolving. However, in order to establish standardized data exchange we have to concentrate on a defined set of accepted and mature technologies and describe the usage in detail. 

This procedure, as part of Section 15 Guidelines, introduces  the common methodology and technology used by the specific business processes described in procedures 2, 3 and 4:

  1. General Data Transport Specifications: describes the applied web service transport protocols and data protocols
  2. Access to the Service: describes some aspects to be taken into account when implementing and installing the services
  3. Communication work flow: covers the communication work flow aspects in a business process session like request, response and data processing.
  4. Dealing with Local Requirements: describes in detail how locally obligatory data elements which are not covered by the ICAR scope can be added to the interfaces
  5. Common Components: describes all the common protocol components which are reused in the specific business processes

Definitions and Terminology

Table 1 contains a list of important definitions for terms and abbreviations used in these guidlelines.

Table 1.  Definitions of Terms used in these guidelines.
Term Definition
TCP/IP Transmission Control Protocol
HTTP Hyper Text Transfer Protocol
XML Extensible Markup Language
XSD XML Schema Definition
SOAP Simple Object Access Protocol
WSDL Web Service Description Language
REST Representational State Transfer
ADIS Agricultural Data Interchange Syntax
ADED Agricultural Data Element Dictionary
JSON JavaScript Object Notation
UNCEFACT UN/CEFACT is the United Nations Centre for Trade Facilitation and Electronic Business
ISO International Organization for Standardization
IANA Internet Assigned Numbers Authority

Scope

gives a pictorial summary of the main elements of this procedure. The numbers in this figure refer to the heading numbers of this procedure.

Figure 1. Scope of procedure 1 of Section 15 of ICAR Guidelines.

General Data Transport Specifications

SOAP/XML Web Services

For the primary data transmission method the W3C standard SOAP/XML version 1.2 has been chosen. Detailed descriptions of this standard can be found here:

http://www.w3.org/TR/2007/REC-soap12-part0-20070427/

Since a “top down” approach has been adapted, ICAR ADE provides WSDL files and a set of XSD files which make up a complete set of machine readable definition files from which SOAP web service implementations can be created. Most modern programming languages e.g. Java, .NET etc. provide tools which facilitate an easy setup process of interfaces classes to be used as a link between data transport layer and business process layer.

For each release of the ICAR ADE specifications a specific set of WSDL/XSD files can be downloaded from the ICAR ADE website – link here. See Table 2. ICAR ADE schema files” for a description of the files. The files will be provided in a zip archive file e.g. “ADE_Schema20150309_1.8.zip” for version 1.8 available here.

The WSDL/XSD files describe different aspects of the transport protocol:

  1. Service end points
  2. Input and output messages
  3. Basic data type definitions (XSD types, UNCEFACT types etc.)
  4. Complex data type definitions : (entity tags, item tags, compositions, attributes, usage of basic data types)
  5. Constraints (data type and value range restrictions)
Table 2. ICAR ADE schema files
File Name /

File Name Schema

File Description
*.XSD XML Schema Definition files, machine readable description of the ICAR ADE XML data structures
ICAR_*.XSD Data element definitions created by the ICAR ADE group
ISO_*.XSD Data element definitions derived from ISO standards
UNECE_*.XSD XSD files provided by the UN/CEFACT project:

(see http://www.unece.org/cefact/xml_schemas/index.html)

UnqualifiedDataType_13p0.XSD XSD file which defines the unqualified UNCEFACT data types. These data types extend the standard XSD data types with additional attributes. In the ICAR and UNCEFACT definitions they are used instead of the standard XSD types.
*_CODE_* XSD files defining code lists
wsMrAde.wsdl ICAR ADE SOAP service description, (WSDL = Web Service Description Language)

It defines a set of message pairs each consisting in a request and a response message.

ADE_v1p8.xsd This file provides the entry point to the ICAR ADE message data structure definitions


Throughout this document messages and data definitions are illustrated using graphical presentations of XSD structures created by the commercial software XMLSPY from ALTOVA.

See for example Figure 2 Example “GetHerdListRequest”

Figure 2 Example “GetHerdListRequest”


Below the symbols used to represent the XSD schemas in the following pages:

Table 3 XSD schema symbols

Alternative Transport Protocols (ADIS/ADED/IsoAgriNet, REST)

Parallel to SOAP other international transport protocols exist. This is for example the ISO ADIS/ADED standard and its last extensions described in ISO 17532(2007).  On the other hand the REST protocol, based on the HTTP transport protocol using XML, JSON and other data encoding technology, has gained a broader acceptance in the domain of internet data transfer during the recent years.

The ADE-WG is working on the development of  a standardized  REST-API description.

ICAR Types and Code Lists

Elements created within the process of the ICAR ADE specification can be identified by the prefix “icar:” within the WSDL/XSD files and descriptions below.

All data items are based on UNCEFACT data types (see section UNCEFACT Data Types)

A detailed description of the data elements is given below.

External Elements

It is the aim of the ICAR ADE standardization process to make as far as possible use of elements already defined within existing standardization frameworks as ISO, IANA and UNCEFACT.

UNCEFACT Data types

The ICAR ADE WSDL/XSD definitions make extended use of UNCEFACT basic types.

UNCEFACT basic types are types derived from basic XSD types. The concept consists mainly of adding further attributes and restrictions to the XSD types. They are defined in file UnqualifiedDataType_13p0.xsd.

In the WSDL/XSD files and descriptions below those elements can be identified by the prefix “udt:

Table 4 UNCEFACT basic data types” lists the UNCEFACT types currently used for the types of a data item.

Table 4 UNCEFACT basic data types
Name Description Comment XSD Type
udt:CodeType A character string that may be used to represent or replace the definitive value. A code is referring to an enumeration String
udt:DateType A particular point in the progression of date Representation as defined by transport protocol String
udt:DateTimeType A particular point in the progression of time Representation as defined by transport protocol String
udt:IDType A character string used uniquely to establish the identity of, and distinguish, one instance of an object within an identification scheme from all other objects within the same scheme. String
udt:MeasureType A numeric value determined by measuring an object Measures are specified with a unit attribute holding a measure unit, a resolution and when it is required a minimum and a maximum value. Decimal
udt:NameType A word that constitutes the distinctive designation of a person, place, thing, or concept String
udt:BinaryObjectType Base64 coded binary data Used e.g. for zip compressed message String
udt:TextType A general text type String
udt:IndicatorType A Boolean indicator or true and false Boolean

UNCEFACT Code Lists

UNCEFACT provides an extensive set of code lists. See Table 5 UNCEFACT Code Lists” for the code lists used by the ICAR ADE specifications.

Table 5 UNCEFACT Code Lists
Code List Description
AgencyIdentificationCode A list of agencies responsible for code list maintenance to be used as value of the attribute listAgencyID in the UNCEFACT data type udt:CodeType
CharacterSetEncodingCode A list of character set encoding codes to be used as value of the attribute encodingCode in the UNCEFACT data type xsd:base64Binary
MeasurementUnitCommonCode An extensive set of commonly used units to be used as value of the attribute unitCode in the UNCEFACT data type udt:MeasureType

ISO Code Lists

See Table 6 ISO code lists for the ISO code lists used by the ICAR ADE specifications.

Table 6 ISO code lists
Code List Description
ISO3AlphaCurrencyCode A list of ISO currency codes to be used as value of the attribute currencyID in the UNCEFACT data type udt:AmountType
ISOTwoletterCountryCode A list of two letter ISO country codes to be used as value of the item Country

IANA Code Lists

See Table 7 IANA Code Lists for the IANA code lists used by the ICAR ADE specifications.

Table 7 IANA Code Lists
Code List Description
CharacterSetCode A list of IANA character set codes to be used as value of the attribute characterSetCode in the UNCEFACT data type xsd:base64Binary
MIMEMediaType A list of IANA MIME media type codes to be used as value of the attribute mimeCode in the UNCEFACT data type xsd:base64Binary

Access to the Service

Exchange Parameters

The equipment should provide the operator with an interface to capture exchange parameters for a given service provider consisting of:

  1. Service provider URL
  2. Authentication information (user/password, token)
  3. Sender ID
  4. Sender name
  5. Sender country
  6. Recipient ID
  7. Recipient name
  8. Recipient country
  9. Type of location
  10. Type of identifier of the location
  11. Identifier of the location
  12. Name of the location
  13. Country of the location
  14. Type of primary animal identifier
  15. Type of secondary animal identifiers (e.g. on farm animal id, animal name)
  16. Other manufacturer or service provider specific parameters

For a given service provider the exchange parameters should be the same for all the consumed services.

Data Transmission

As the services are not permanently accessible, no request should be sent by the equipment before having checked if there is access to:

  1. The network
  2. The service provider server
  3. The service itself


Any new data should be registered by the information system as soon as possible.

No new data should be sent to the information system as long as previous data is still being processed. The service consumer has to wait for a processing result or a ticket in case of asynchronous processing.

No ticket should be sent to the information system before the time given by the information system.

Any message whose syntax is not in compliance with the syntax requirements should not be sent.

As soon as a data is included in a request an answered by a response it should be considered as sent to the information system.

Service providers and consumers should be prepared for updates and deletes. The service provider should take also take into account that the sending party is declaring updates as a create operation not knowing that the data is already present in the service provider’s data base. (e.g. in case of retransmission initiated by the consumer)

Preparing a request in compliance with business requirements should be performed separately from preparing a message whose syntax depends on the type of technical mapping (web service with SOAP at the moment).

Encryption

If possible encryption should be used in order to insure privacy and authenticity. The process of encryption itself is not described here. The encryption capabilities of the applied transport protocol should be used e.g. HTTPS for HTTP-Transport.

Compression

In order to minimize the transport delay of big messages, especially when coded in xml, compression should be applied. Some transport protocols offer compression features like transparent gzip compression of http responses. However since http compression is only applicable to the response it is not usable for the compression of data loads in http requests.

In order to avoid this limitation a special XML based compression mechanism is described for ICAR data exchange. Specific messages can be defined as zip compressible offering the alternative of uncompressed or zip compressed Base64 binary encoded xml structures both for request and response. For details see chapter “Message Design” below.

User Right Verification

The right of a given user to use a given service is verified by either using an “authentication token” which prevents him from sending a user ID and a password at each request or by a user name authenticated by a password.

Authentication by user name and password should only be used with encrypted transport.

Communication work flow

Data Processing

Any service may be provided either in real time or in time differed.

A response should be sent to any response within several seconds.

Any request and any response should be stored by the equipment and the information system.

The requirements to update the data base of the information system differ from one service provider to another one.

Any request which is not in compliance with one the following conditions should not be processed:

  1. Compliance with the syntax
  2. Validity of the identification token
  3. Validity of the ticket if any

Request Specification

Any request should consist of three parts:

  1. A single message header (see the entity MessageHeader of Type “ADEExchangedDocumentType” in  ‘Data description’) The sender of the request creates and provides his own unique MessageID (Item ADEExchangedDocumentType.Identifier). See chapter “Best Praxis for MessageID Creation” below.
  2. A single standard request (see the entity StandardRequest in ‘Data description’)
  3. A specific request which may consist of a specific message for a given service or the zip compressed base64 binary of the specific message or a ticket. A ticket is sent in order to retrieve the results of an asynchronous message processing task. (see diagrams below)
Figure 3 Request with a specific request


Response Specification

Any messsage response should consist of three parts:

  1. A single message header (see the entity MessageHeader of Type  “ADEExchangedDocumentType” in ‘Data description’) The sender of the response creates and provides his own unique Message id. (See chapter “Best Praxis for MessageID Creation” below)
  2. A single standard response (see the entity StandardResponse in ‘Data description’. In the standard response the original MessageID from the sender is returned in the item RequestID. The standard response contains the item RequestProcessingStatus which identifies the status of the request processing on the side of the service provider:
Table 8 RequestProcessingStatusCodeType (base xsd:string)
Key Description
O Processed without errors
P Data accepted for asynchronous processing (client can retrieve the processing result later using provided ticket)
E Processed and rejected due to errors
W Processed with warnings

c. A specific response which may consist of either a ticket or specific message for a given service or the zip compressed base64 binary of the specific message.

The existence and content of the specific response depends on the definition of the service and the content of the RequestProcessingStatus item.

In case of RequestProcessingStatus = ‘P’ the specific response holds a Ticket

If a specific response message is defined it can either be represented uncompressed or zip compressed of type Base64binary.

If RequestProcessingStatus = ‘P’, ‘E’ or ‘W’ there might be no specific response at all depending on the message definition.

Figure 4 Response with a Specific Response
Figure 5 Specific Response Schema
Figure 6 Standard Response


Best Praxis for Message-Id Creation

The message ID created by both service consumer and service provider should be globally unique in order to allow a simple identification of each message e.g. for tracing and debugging purposes.

The MessageID should be based on a chain of unique hardware identifier, a unique software identifier and an additional distinguishing component large enough to uniquely identify messages created at very fast rates. This dynamic component could be a timestamp with ms precision or a sequence. The unique hardware identifier could be the globally unique MAC address of the network interface. The unique software identifier should be unique on the system running the software.

There are tools in different programming languages which provide simple means of unique id creation e.g. Guid.NewGuid() in C#

Dealing with Local Requirements

The ICAR ADE standardization process cannot take into account all local requirements. In the first place it offers a common basis to build on.

In the first place “local” is a placeholder for national standards. However it is also possible to add local extensions based on an agreement between a service consumer and a service provider (e.g. for special projects, prototypes etc.).

In order to allow local extensions and to fill standardization gaps which cannot be covered by ICAR the ADE specifications provide two ways of local variations and extensions:

  1. Local Code Lists: Local code lists for all items of type udt:CodeType for which ICAR is not yet able to provide a commonly accepted code set
  2. Local Additional Data: An optional list of key value pairs in each specific entity to be used for locally required data elements

Local variation should be reduced to a minimum since it is contradictory to the goal of ICAR to establish global standards, leads to increased complexity, complicates software development and creates additional costs.

2.1        Local Code Lists

Local code lists must be provided for items of type udt:CodeType in the context of a local standardization process, e.g. breed codes.

The content cannot be checked with the SOAP/XML validation procedure but must be evaluated in an extra program step before filling the XML data structures and after extracting the data from the XML structures. It is the responsibility of the software developers to assure that the data exchange is in accordance to ICAR and to local specifications.

In order to help manufacturers with the maintenance and integration of local code lists these specifications define the service GetLocalCodeList which can be used to query local code lists from the local service providers. For details see section “Technical Services”.

Local Additional Data

Sometimes it is necessary for local implementations of the ICAR messages to add local data items to the specific messages which are not known or not covered by the ICAR specification process.

For those scenarios a simple and flexible extension (LocalAdditionalData) has been added to each specific entity.

It consists of an optional list of code/value pairs of type udt:textType which can be filled according to local definitions.

It is the responsibility of the local business process owners to document and assure the proper handling of this code/value pairs. The ICAR wsdl/xsd definitions only check the proper usage of the XML structure not the content.

For details see the wsdl/xsd files and the section

“CommonComponents”-”LocalAdditionalDataType”.

Common Components

This part describes common components which are used in all the specific services described in anex B, C and D.

ADE Exchanged Document Type

The entity ADEExchangedDocumentType (Figure 7 ADEExchangedDocumentType) gives information describing general aspects of the message.

Used by data element: MessageHeader

Figure 7 ADEExchangedDocumentType
  1. Identifier: Unique identifier of the message given by the sender
  2. Issueing: The date and the time where the message is issued.
  3. Version: Version number of the message
  4. Language: Language used for the message
  5. LocalAdditionalData: Optional list of key/value pairs used in a local context as described in section “Common Components – Local Adaptions”
  6. SenderParty: Organization or person responsible for the content of the message, not a server identifier.
  7. RecipientParty: Organization or person responsible for processing the message, not a server identifier.

ADE Party Type

The entity ADEPartyType (Figure 8 ADEPartyType) gives the name and the identifier of a party which may be either the sender or the recipient of a message.

Used by data elements: SenderParty, RecipientParty

Figure 8 ADEPartyType
  • Name: Identifier of the specified party
  • ID: Identifier of the specified party
  • Country: Country of the specified party in accordance to code list ISOTwoletterCountryCodeIdentifierContent

Standard Request Type

The entity StandardRequestType (Figure 9 StandardRequestType) gives the standard content of a request.

Two different types of authentication can be used, token or user/password based.

Used by data element: StandardRequest

Figure 9 StandardRequestType
  1. AuthenticationToken: temporal code used for verification of the user’s identity and of its right to use a service.
  2. AuthentificationLogin: Pair of username and password used for login
  3. UserName: User name used for authentication
  4. Password: Password used for authentication

Standard Response Type

The entity StandardResponseType (Figure 10 StandardResponseType) gives the standard content of a response.

Used by data element: StandardResponse

Figure 10 StandardResponseType
  1. RequestProcessingStatus: Status of request processing returned by server according to code list RequestProcessingStatusCode
  2. RequestID: Copy of the message identification in the request’s MessageHeader.Identifier item
  3. RequestProcessingError: In case an error happens during the processing of the request an error description is provided here

Error Type

The entity ErrorType (Figure 11 ErrorType) gives the request processing errors.

Used by data element: RequestProcessingError

Figure 11 ErrorType
  1. ErrorID: Identifier the error, e.g. error text message, local error code etc.
  2. ErrorSeverity: Severity of the error according to code list ErrorSeverityCode
  3. ErrorDescription: Human readable description of the error as text

Ticket Response Type

The entity TicketResponseType (Figure 12TicketResponseType) conveys a ticket in order to retrieve a response processed in time differed (asynchronous processing mode).

It is used in SpecificRequest and SpecificResponse by data element TicketResponse.

Figure 12TicketResponseType
  1. TicketID: Identify the response to retrieve
  2. NotBefore: Date time of probable response availability

ZIP Message Type

ZipMessageType (Figure 13 ZipMessageType) is a wrapper type designed to transport the zip compressed version of the specific data part of a message. Contrary to http compression method which can only be used for the response part this method can be applied to request and response. It is defined as a choice for each specific request or response which transports data entities.

Figure 13 ZipMessageType
  1. ZIPMessage: zip compressed data

Local Additional Data Type

The entity LocalAdditionalDataType (Figure 14 LocalAdditionalDataType) is a container for code value pairs for local usage outside the ICAR specification scope.

It is used as an optional list element LocalAdditionalData in each specific entity.

Figure 14 LocalAdditionalDataType
  1. AdditionalDataCode: Name of the code value pair
  2. AdditionalDataValue: Value of the code value pair
  3. AdditionalDataComment: Optional description of the local data

Time Period Type

The utility type TimePeriodType (Figure 15TimePeriodType) is a wrapper type designed to transport the beginning and end of a time period.

Figure 15TimePeriodType
  1. StartTime: The start of the time period
  2. EndTime: The optional end of the time period

Naming Rules

Names of data elements, operations and messages use upper camel case.

Service name consist of:

  1. A prefix which gives the type of operation according the following:
    • Create: Insert data into a data base
    • Update: update of a data base
    • Get: retrieve data from data base
    • Delete: delete data from data base
  2. The name of the service For example UpdateMilkingResult is the service which allows updating a data base from milking results exchange. Message name consists of:
    1. The name of the service
    2. A suffix, gives the type of message:
      • Request
      • Response

For example, UpdateMilkingResultRequest is the request to trigger the service UpdateMilkingResult.

References

1.       Semantics for Smart Dairy Farming: a milk production registration standard – SDF June 2013

2.      UN / UNCEFACT Modeling Methodology User Guide (CEFACT / TMG/N093)

3.      UN / UNCEFACT Business Requirements Specifications Document Template (CEFECT/ICG/005)

4.      ISO 11787 : Electronic data interchange between information systems in agriculture —Agricultural data interchange syntax

5.      ISO 11788 : Electronic data interchange between information systems in agriculture — Agricultural data element dictionary —Part 1: General description —Part 2: Dairy farming

6.      ISO 17532 : Stationary equipment for agriculture —Data communications network for livestock farming

7.       ISO 11784: Radio frequency identification of animals - code structure

8.      ISO 3166 -1 : Country code

9.      ICAR G