View source for Conformation Recording: Improving data quality and monitoring classifiers


== Introduction ==
When collecting data on animal performances on a routine basis it is important to do this in a consistent and transparent way. In this way quality of data can be guaranteed and for everybody it is clear how it is done. This is also important for scoring animals for conformation traits, which is normally done by classifiers, specially trained doing this job.

This chapter describes the improvement of quality and transparency of data collection for conformation traits.

== Practical aspects of type classification system ==
One organisation should be in charge of classifications within each evaluating system.

There should be a head-classifier in charge of training and supervising other classifiers within the evaluating system to achieve and maintain a uniform level of classification. Additionally the exchange of information between head-classifiers from different systems/countries is recommended.

Individual full time professionals should complete classification. Classifiers should be independent of commercial interest in AI-bulls/studs.

Classifiers must record the trait as observed without adjustment e.g. Age, stage of lactation, sire or management system.

The working information provided for the classifier should make no reference to the pedigree or performance of the animal.

Classifiers should always rotate classification areas (herds and regions) to ensure a good data connection between regions and to minimise the sequential scoring of animals by the same classifier. This way of working reduces this risk of classifier times regional genetics interaction or classifier times herd interaction.

An advisory group can be installed with expertise in the field of conformation classification, statistics, breeding, training people, in order to monitor and advise on the improvement to the classification system.

All factors accounting for any non-genetic variance should be recorded when a herd is visited, e.g. classifier's identification, date/time of scoring, management group, housing system, flooring, nutritional status. This makes it possible to find possible interactions between the environmental factors and the trait scored.

Types of housing can be free stall, tie stall, mixture (stall plus outside).

Types of floors can be concrete, cement with groves, slats, sand, rubber, straw, pasture.

== Training and monitoring of classifiers ==
The monitoring and performance evaluation of classifiers is an important part of the standardisation of the ICAR international type program.

''Objectives''

Improve accuracy of data collection, within country all classifiers should:
<ol style="list-style-type:number">
<li> Apply the same trait definition</li>
<li> Apply the same mean.</li>
<li> Apply the same spread of scores.</li>


</ol> 


''Tools for objective 1:''

# National group training sessions.
# Statistical monitoring of individual classifiers performance with reference to mean, spread and normal distribution of scores.
# Compute the correlation between the scores of one classifier and the group by using bivariate analysis. This shows the quality of harmonisation of trait definition between classifiers.
# Improve the genetic correlation for linear traits between countries (Interbull evaluation)
# Apply the same trait definition in all countries.

''Tools for objective 2:''

# International training of head classifiers.
# International group training sessions.
# Audit system.

If a country decides to change the definition of a trait, it is recommended not to use previous scores or use only as a correlated trait in the national genetic evaluation system.

=== 1.3.1 National group training sessions ===
One way of improving harmonisation of scoring by classifiers is having regular training sessions with a group of classifiers.

There are many ways to accomplish trait harmonisation through training sessions. Normally a training session consists of scoring a group of animals and the scores of individual classifier are compared with the scores of the other classifiers and/or head classifier.

Attention points are:

# Use a group of animals for training session which is representative for the population classifiers have to score during their herd visits.
# Deviations of individual scores are discussed and it is made clear which is the correct score for a certain trait on an animal.
# Scores of each classifier are analysed per trait using some analysis tools:
#* Compute the mean and standard deviation of the deviations of the scores on animals per trait, per classifier. The deviation is the difference between the score and the average group score for a trait, for an animal. This gives insight in the scoring of individual classifier: always above or below the mean, more variation in scoring a trait than the group/head classifier. (with a test it can be shown if the differences found are significant).
#* Compute the spread of the deviation of scores given by classifier per trait. This gives insight in how consistent a classifier is scoring a trait. (with a test it can be shown if the differences found are significant).
# Instead of scoring a group of animals once, the animals can be scored twice by the classifiers, for example in the morning and in the afternoon. Based on these scores (approximately 20) the repeatability per classifier per trait can be computed.

=== 1.3.2 Statistical monitoring of individual classifiers ===
The scores of a classifier from a certain period in time can be analysed. A period can be 12 or 6 months, for example.

From these scores the mean and standard deviation can be computed. The mean should be close to (maxscore-minscore)/2, and the standard deviation should be near (maxscore-minscore+1)/6, where minscore is the lowest score on the scale and maxscore is the highest score on the scale. For example: scoring a trait on a scale of 1-9, a mean is expected of 5 and a standard deviation of 1.5.

Another option is to compute the correlation between the scores of one classifier and the scores of rest of the group by using bivariate genetic analysis. This shows the quality of harmonisation of trait definition between classifiers (Veerkamp, R. F., C. L. M. Gerritsen, E. P. C. Koenen, A. Hamoen and G. de Jong. 2002. Evaluation of classifiers that score linear type traits and body condition score using common sires. JDS 85:976-983).

For this analysis, two data sets are created, one with scores of one classifier and the other with scores of all other classifiers from a certain period, for example 12 months. Both data sets can be analysed in a bivariate analysis, estimating different (genetic) parameters. The analysis can be carried out for each trait and for each classifier. From the bivariate analyses the following parameters can be derived:

# Heritability: the heritability estimated within each classifier can be used as criteria for the repeatability of scores within classifiers, albeit the optimum value is not unity but depends on the true heritability of each trait.
# Genetic correlation: the genetic correlation between two data sets can be used as a measure of the repeatability between classifiers, where a genetic correlation of one between classifiers is expected.
# Genetic standard deviation.
# Phenotypic standard deviation (= square root of genetic variance and error variance).

For the evaluation of each trait for each classifier the diagram in Figure 1 can be used.

Evaluation obviously starts with the mean score for each classifier, i.e., the mean should be close to the trait standard (5 for linear traits and 80 for descriptive traits). Secondly, the genetic standard deviation should not be lower than the average.
[[File:Scheme for evaluation trait by classifier using genetic parameters.png|center|thumb|''Figure 1. Scheme for evaluation trait by classifier using genetic parameters.'' ]]




''Figure 1. Scheme for evaluation trait by classifier using genetic parameters.'' 


If the genetic standard deviation is lower, this could be due to the scale used (measured by the phenotypic standard deviation), due to poor within classifier repeatability (a low heritability) or both. If the low genetic standard deviation goes together with a low phenotypic spread, the advice is the classifier should use the scale in a better way, use more the extreme scores. If the genetic spread goes together with a low heritability, then the classifier should score the trait more consistently, apply the same definition.

If the genetic correlation is too low the classifier is likely to score a trait different than other classifiers.

All the parameters from the system can be tested using the standard error on the parameters estimated. Every classifier can be tested against the average of the parameters of all classifiers for a certain trait. A classifier with a few scores may deviate a bit more from the average of the group, therefore taking the standard error into account in a statistical test is more fair.

== Auditing a classification system ==
The Classification system applied can be further improved by using an audit system where experts familiar with the conformation classification in other countries or organisations, examine the situation in your organisation or country.

An important issue is that information is exchanged between people responsible for the classification system.

Different options to audit are:

# By using international workshops, in which information can be informally exchanged regarding how classifiers are trained and conduct their daily work
# By inviting classifiers and/or a head classifier from another country or organisation to participate in or lead group training sessions
# By having a group of experts visit an organisation responsible for classification, conduct a survey on methods and procedures, report their findings and makes suggestions for improvements.