Full Report on Proposed Rule Changes in the Judging Method of Dressage

Dressage

FEI Dressage News

FEI Dressage Committee Chairman Frank Kemperman submitted the following full report of the proposed rule changes in the judging method of dressage.

This report was sent to and published by the FEI with the memorandum date of 8 September 2010. Eurodressage has reproduced this public document here for your convenience. A summary can be found here

Background & Objective

Our Dressage sport has significantly increased its popularity within the last years, with growing interest of the public, the media, as well as sponsors. Whilst this is proof that the sport itself is attractive, its character as an aesthetic sport is naturally causing a contentious issue with regard to judging from time to time. There is no doubt that the education of the judges and their ongoing assessment according to fixed standards is of the utmost importance to ensure the highest level of fairness and correctness in the sport. Beyond that, the nature of our sport requires a continuous willingness of everyone involved to increase its transparency and to remain open-minded for modernization without forgetting the tradition and history of our sport. Whilst the judges are an important pillar in the system of Dressage competitions, we also have to be eager to provide those judges with working conditions that don’t only enable them to judge in a fair manner, but which also take into account that human beings can make mistakes which should ideally not affect the result of an athlete.

The rule changes that are proposed in this paper are the result of a detailed analysis on the fitness for purpose of our current judging method in Dressage and are aiming to achieve a maximum of transparency, fairness and, as a consequence, trust in our sport.

Strategy

At its meeting in November 2008, the FEI Bureau appointed an FEI Dressage Task Force (DTF) in order to look into specific issues within the Dressage sport. The appointed members of this DTF came from different stakeholder groups, such as organizers, trainers, riders, judges, chef d’equipes and horse owners. One of the tasks given to them by the FEI was “The fitness for purpose of the method of judging Dressage competitions needs thorough review – both in terms of the number of judges, their positioning and the judging process.” During their one-year-term until the General Assembly in November 2009, the DTF had regular physical meetings as well as conference calls and also carried out a “Judging Systems Trial” in Aachen (Sept 2009). At this trial, several judging methods had been tested by simulating a big tour and a small tour with around forty rider/horse combinations. As a conclusion of this trial, the DTF had proposed to the General Assembly in 2009 to test the following four judging methods as a pilot project in 2010:

7 judges (instead of 5)
Half Marks
Judges Supervisory Panel (JSP)
Dividing of Tasks in Freestyle Competitions

Implementation

During the season 2010 the above mentioned judging methods have been tested under real conditions at the following shows:

CDIO5* Rotterdam 16-20 June 2010 (7 judges, JSP, dividing of tasks)
CDI3* Vidauban 18-20 June 2010 (7 judges, half marks, dividing of tasks)
CDI4* Fritzens-Schindlhof 25-27 June 2010 (half marks)
CDI2*/3*/CDIJ St. Petersburg 25-27 June 2010 (half marks)
CDIO5*/CDI4*/U25 Aachen 13-18 July 2010 (7 judges, half marks, dividing of tasks, JSP)

After each of those events the FEI has received detailed information from the Foreign Judge, show organizer as well as riders, trainers, officials as well as own FEI staff that has been present at the shows in Rotterdam and Aachen. In addition to those personal impressions, Dr. David Stickland and Dr. Wojtek Markowski have conducted statistical analyses on those pilot projects to find out if/what those different judging methods would change compared to the current judging system.

Conclusion

Referring to the gained experience in this year of pilot testing, the FEI Dressage Committee (DC) will propose to the General Assembly 2010 to implement the following elements as a supplement to the current Dressage judging system:

7 judges (instead of 5) for defined events
Half marks for all events
Judges Supervisory Panel (JSP) for defined events

The following judging elements, if approved by the General Assembly 2010, would have to be implemented as follows:

7 Judges (instead of 5)

The DC proposes to use the 7 judges for Olympic Games, World Equestrian Games, Continental Championships on Grand Prix level as well as World Cup Finals. Using 7 judges would also be allowed at other shows if the organizer wishes to do so. All 7 judges’ scores would count and the two additional judges have to sit on the quarter lines on the short side of A. Cost-wise, the World Cup Final would have to add two judges. For the other events, 7 judges are already appointed, and an extra reserve is needed.

[Note: Only if this was not possible space wise, with FEI dispensation, could the two additional judges sit on the long sides, 10 meters from the corner of the A-side.]

The statistical reports have shown that using 7 judges gives a more correct total result. This would also lower the influence of each individual judge from 20% to 14%, i.e. reduce the influence of each individual judges’ marks and decrease the risk for nationalistic judging. The statistics have also shown that the use of 7 judges does change the rankings. In some tests where two additional judges were taken out, the rankings would change for approx. 50% of the participants (changed placings for 21 out of 38 combinations). If the highest and lowest scores were taken out, the placings of 10 out of 38 still changed. Taking out the highest and lowest scores does not achieve the objectives of consistency and reducing the effect of a given potential instance of biased judging.

The tests in 2010 have shown that having 5 judges on the one short side at C (tested in Aachen) does not add much as they all see such a similar angle. Having the two extra judges on the other short side at A on the quarter lines was seen as positive, as it added a different view (360 degrees by the 7 judges). It was also tested to add two judges on the long side which was not ideal.

Half marks

The DC proposes to permit the use of half marks in all FEI Dressage competitions in order to give the judges the opportunity to refine their judging and make it more precise. Half marks can be given in their full range from 0.5 up to 9.5 and for all kind of judgment (marks for movements, collected marks, technical part in the Freestyle both per movement and as final mark for that movement, artistic part in the Freestyle).

In general the use of half marks showed no change in the ranking but somewhat higher marks (average 0.5%). On average the marks seem to go up with half marks, however with the tendency of increasing for the stronger riders and decreasing for the weakest ones. The use of half marks needs no extra education and could be useful because a “small 7” or a “good 8” makes no sense for the riders. No extra costs will be had.

Judges Supervisory Panel (JSP)

The DC proposes that JSP should be mandatory for the same events as the 7 judges (Olympic Games, World Equestrian Games, Continental Championships on Grand Prix level and World Cup Finals).

One purpose of the JSP is to have a kind of a “back-up” for the judges in order to protect both judges and riders by preventing unfair marks. A judge may get distracted for some reason or is not able to see a mistake as a result of his view which could then be corrected by the JSP. However, there are very strict rules to what the JSP is allowed to correct which would, for the time being, only enable them to correct factual errors (only clear, definite technical mistakes and counting errors). It has also been agreed on that the JSP would not be able to set marks outside the range of the judges’ given marks. Those rules are clearly defined and put together in a guideline, based on the experience gained during the pilots.

Seated in a position which gives good overview of the entire arena and provided with computer screens which show the marks and running average of each judge simultaneously as soon as the judge enters his mark, the JSP members are enabled to see all given marks in real-time. Additionally, all rides are videotaped which gives the JSP the possibility to rewind real time and thus making sure that their correction is justified.

The JSP will have the possibility to lower marks (if a judge has missed a clear mistake) but also to rise marks (if the JSP is sure that a rider has been punished for a mistake which did not happen). Furthermore the JSP will inform the judges immediately after the competition which marks have been changed and will supply input for the debriefing of the respective competition. Notwithstanding it is the task of the President of the Ground Jury and the Foreign Judge to actually lead the debriefing.

Apart from supervising at events, the JSP should evaluate the quality of judges in general, make a proposal for assessment of judges and their status (promotion), come with a “long list” proposal of judges (for the Olympic Games, WEG, Continental Championships on Grand Prix level, World Cup Finals) and provide a general overview of the “judges’ world” to the FEI Dressage Committee and the FEI Headquarters, as described in the Dressage Task Force Report.

Smaller events could benefit from having a JSP as well, whereas this is both a financial and organizational issue. The cost for the JSP at the shows where its presence is mandatory will be added to the contract. If the JSP was to be sent by the FEI to 2-3 other shows, the cost would be carried by the FEI. The JSP team for the non-mandatory events should ideally have equipment to correct the given marks but should at least take part in discussions after the competition.

The ideal composition of a JSP group (3 members at a show) would be 2 judges and 1 trainer or rider. The judge must be an independent and experienced FEI 5* judge, the trainer/rider preferably a “fellow”. All must be well respected and diplomatic, must be prepared to travel to shows, follow the judges’ codex, have integrity as well as communication skills. Active judges could be on the JSP but would then need to take time off from judging on an international level. Similarly, trainers would take time off training and riders off riding on an international level in order to avoid any perceived conflicts of interest. There are not only three but a pool of people needed to always have a JSP available if needed. Furthermore, the trainer/rider should attend 5* judges forums. Geographical spread among the JSP members is a plus and there can be max 1 person per National Federation in the JSP, however the most important thing is to have the most experienced and respected people in this team. No age limit would apply. The time limit for being on the JSP would be 2x 2 years (the minimum 2 years), with a rotation system to ensure that experience and knowledge can be transferred.

If the General Assembly 2010 approves this proposal, a detailed job description and the tasks for the JSP would be set and the stakeholder groups (officials, riders and trainers) would be asked for their input. Based on their input the FEI Dressage Committee in conjunction with the FEI Dressage Director would decide about the appointment of the JSP members.

[Note: The dividing of tasks in Freestyle tests will not be proposed to the General Assembly as the FEI has received no positive feedback regarding this throughout the testing period. The judges have reported that a dividing of tasks doesn’t work in practice as the judges who were only giving marks for the artistic part still had to assess the technical part in order to give fair and correct artistic marks. Also, the actual number of marks given for the rider is reduced by half when dividing the tasks.]