Wayne Channon: Disaster Waiting to Happen

Wayne's World

IDRC secretary-general Wayne Channon is joining the debate on Eurodressage about the popularization of the dressage sport through the improvement of the judging system Responding to Stephen Clarke's letter to the editor on the high-low score scratch and David Stickland's proposal for a detailed "HiLo Drop"

, Channon shares his view on what can improve the sport until a code of points is put in place. Channon first publicly advocated his Code of Points idea here on Eurodressage in 2009.

Disaster Waiting to Happen

Michael Klimke asserted that we should drop the highest and lowest judges’ scores. Stephen Clark made a strong case for retaining both. Who, if either, is right? I hope Stephen will be delighted to hear that I agree with him. However, I also agree with Michael. Read on and you will see how I manage to reconcile both viewpoints.

Stephen’s case?

In essence, Stephen’s argument is:

The difference in judges' marks is solely due to different viewing positions.
Statistically, dropping high and low scores makes no difference, so why do it?
Psychological effect - it would make judges too careful; they would move to 6.5s and we won’t see the 10s.

This looks like a reasonable argument to me but I think there are some flaws.

Differing viewing positions - different view positions do have an effect but they do not account for the significant differences we have in scoring. Look at Herning with one judge 7% lower than the average for Delphi - this was the European Championships and one judge changed the medals. Unfortunately, this was not a one-off, it happens all the time. We need a solution that corrects for a judge that is wrong but does not change the majority view of the test. What I am really saying is that until we have a Code of Points, we need a sensible band-aid.

Statistically it does not make a difference - Stephen is right. Statically, on the vast majority of rides, taking away the lowest and the highest scores would not make a difference. And any new correctional system must not make a statistical difference. We want the system to correct ONLY those scores that are wrong. Such as Delphi’s.

Psychological effect - I don’t think Stephen would argue against any correction to errant judge scores, as he has already introduced a correction system for judges that are too high or too low (see Article 438 below). As his system corrects up or down, the implication is that he believes that it is dropping a judge's whole score that would be deleterious to their mental health.

The further, and extremely worrying implication of this statement is that he is saying that judges don’t give marks based on quality of execution. They have to be “confident enough” to give high scores. Confident? I thought if I rode for a 10, I would be given a 10. Surely, he cannot mean this.

If our system of judging is as good as Stephen believes, then surely judges will mark according to what they see. If indeed we have to be careful of the mental health of our judges due to the impending threat of having their score deleted, then maybe we should move to a Code of Points where we don’t have to entertain such notions.

The current rule - a disaster waiting to happen...

It is worth examining Stephen’s current rule which was adopted following the debacle at the European Championships:
Article 438 If a judge’s final score for a Horse/Athlete combination varies (above or below) by six (6)% or more from the average of the scores of the other judges for the same combination, the JSP may, by unanimous decision, change that particular score to be the same as the next closest score.

In my view, this is not just a poor solution, but a disaster waiting to happen, for many reasons. Principally:

1. If a judge is 6% lower than the rest, they are not competent to be judging that test. Why should their score be adjusted so that it double weights the next lowest (or highest)? It suggests that they were correct but just a little on the low (high) side. When in reality, they are plainly wrong and their score should not count. The same applies for if they are on the high side.

2. I am not saying anyone would, but this is a recipe for nationalistic/personal favourite judging - just go as high as you want and you will then ensure that you score will be adjusted to the highest of the others - and similarly if you wanted to sabotage a competitor.

3. There can’t be an arbitrary cut-off. Imagine if we had a judge that was always 5.9% or 6.1% lower for all competitors, so at least they were consistent. It could happen that they were 6.1% lower for one possible medal winner and 5.9% lower for the other. One of the scores would be corrected and the other would not. This would give the medal to the overall lower scoring combination. Just wait for the outcry on that one!

4. What is magical about 6%? If someone is 5.9% lower, it does not mean their mark is acceptable. It is still poor judging.

5. It is at the discretion of the JSP. This means they may choose to enforce it for one person and not for another, at the same competition. How is that fair?

The way forward

For each movement, we should drop the highest and lowest judges’ scores - this is what David Stickland called HiLoDrop (Figure). This is very different from dropping the highest and lowest scoring judges’ marks for the whole test.

The benefits of this approach are:

No judge’s whole score will be dropped. This addresses Stephen’s concern for their mental health.
It will not affect the overall score of the vast majority of tests.
It will not force judges to give a 6.5.
It will still allow judges to give the 10s and 9.5s - knowing that if they and their colleagues did so, the result would not be watered down by a negative ninny.
It will stop nationalist judging - it would now take 2 judges to vote nationalistically.
The score will become more consistent.

So there you have it. I agree with Stephen and Michael - just a small Stickland twist to both their systems and the sport will be the better for it. At least until we have a Code of Points.

by Wayne Channon for Eurodressage