How not to judge: Part 1

By Srdjan Miletic

Over the past six months I’ve heard regular complaints about the quality of judging on the Dutch circuit. This isn’t to say that the quality of judging is bad, just that it could be better.This article contains a few choice issues people have come to me with and how they should be resolved.

How to judge

http://e1o2.blogspot.nl/2015/01/how-to-judge-unpacking-persuasiveness.html

How not to judge
1.Judging based on “What’s left standing”

The Problem: Judging teams based on what arguments remain unrebutted at the end of a debate is unfair. The problem with this approach is that a team with stronger argumentation may lose to a weaker team due to their opponents choosing to focus on rebutting their arguments. For example, 1st Prop may lose to 2nd Prop despite bringing stronger argumentation simply because Opp chose to focus on rebutting closings arguments, leaving less of them standing at the end. This is arbitrary, unrelated to a team’s actions and hence unfair. Also, it is strange that a team which brings worse arguments can win over a team with better arguments.

The Solution: Judge by allocating credit for bringing and attacking arguments. Do not remove credit from a team if their argument happens to be rebutted

2: Giving imprecise feedback

The Problem: As a judge, your call should clearly explain which team won and why. Talking about this in terms of who brought more “analysis” or “engagement” is likely to be too abstract to be useful. It makes it difficult for teams to understand why they won or lost and hence how to improve. It also encourages sloppy thinking and bad calls as a poorly explained call is difficult for teams to scrutinize or argue against.

The Solution: When giving feedback go through the teams either chronologically or, preferably, from first to last. For each team clearly explain what arguments you believe they brought and which of their opponents arguments they successfully rebutted. Explain how persuasive you find each argument and rebuttal, justifying this from the perspective of the average informed voter, and hence how persuasive you find their team as a whole. Then explain why they were more or less persuasive than the team ahead of them by comparing arguments. For example:

Bad judge: Second proposition wins because their analysis and impacting was better than second oppositions.

Intermediate judge: Second proposition wins because their analysis shows that an invasion would lead to less atrocities than a bombing campaign and I care more about atrocities than the terrorist attacks second opposition talked about.

Good judge: Second proposition convinced me that an invasion of Syria would lead to a faster victory than a bombing campaign and hence a shorter war and less deaths from ISIS atrocities. That wins over second opposition whose point on another invasion hurting the image of the US and hence encouraging terrorist attacks, while plausible, seems to me to be less important than preventing atrocities which I, as the average voter, assume would kill a far greater number of people.

3: Judging extensively analyzed or sophisticated arguments as persuasive

The Problem: .Just because an argument has a great deal of analysis and that analysis is complex does not mean that it is persuasive. An argument is persuasive insofar as an average person would see it as true and important. If incredibly complex analysis does not make an argument very plausible or important, it is not a persuasive argument.

The Solution: The amount of analysis an argument has doesn’t matter! What matters is the extent to which that analysis would convince the average informed voter that the argument is true and important. Never be tempted to give an argument greater weight just because there was more reasoning or argumentation behind it.

4: Judging a team as off-clash because they choose a different burden from other teams

The Problem: Just because three teams in a debate choose to focus on one type of argumentation or one burden does not mean that a team which chooses a different burden or area is in any way doing something wrong or failing to engage. For example, in the debate “THW organize simultaneous in or out referenda in all EU member state”, say three teams in a debate have focused on why referendums and the EU are good or bad and the fourth chooses to focus on why, if held, referendum should not be simultaneous. As prop has to show that there should be referendums and that they should be simultaneous, this team is on-clash and should not be penalized in any way for choosing a different burden.

(Note: The above motion is taken from Rotterdam Open 2015 but the example itself, while somewhat based on, is not intended to represent the debate I was in. In that debate, while I do believe that the call could have been better explained, I do not believe that it was unfair or that it was necessarily based on the above reasoning. TL:DR: No offense intended, it’s just an example)

The Solution: A team is on clash as long as their arguments are both true and important as reasons for doing or not doing the motion. The fact that they choose a different direction from other teams in the debate does not make their reasoning and more or less persuasive.

Mascha Bloemer

ASDV Bonaparte | + berichten

Mascha is een alumnus van de Amsterdamse Studentendebatvereniging Bonaparte. Zij was redacteur van SevenTwenty (2012-2013) waarna ze in 2013-2015 de rol van hoofdredacteur op zich pakte.

1 reactie tot nu toe

WiegerGeplaatst op2:09 pm - feb 6, 2015

1) An argument that is left standing has some merit over an argument that has not been left standing. The problem is that the strength of an argument is relative to the debate. A judge does not know what the inherent merit is of an argument beforehand. Whether or not is has been rebutted in the debate can be an indication as to the inherent strength of the initial argument. Is this the only factor that contributes to the inherent strength? Ofcourse not. But if two equally analysed arguments are brought, but one is more succesfully rebutted than the other, this is an indication that this is the weaker argument.

2) I agree that feedback should focus on clashes between (i) arguments and (ii) teams. Two remarks about this though. a) a judge does not always have the time to go into specific analysis that was relevant in the tradeoff. This is why you should always go to the judge after the debate if you suspect he did not thing about this. b) it is my experience that even the best judges are ignored in this if a team does not agree with the judgement (I am guitly of this as a participant as well, which I shall admit only once. Also, this is of course not the case in the instance that you are remembering right now [reader]).

3) The amount of analysis DOES matter. First of all, if two arguments are equally inherently persuasive, the team with the most analysis on the argument wins as they “explained” it better. Even if the arguments are not equally persuasive, the “average newspaper reading voter” could be persuaded more by a very well explained argument as opposed to a poorly explained brilliant argument. Furthermore, a well explained argument normally has a broader analytical base to support its end conclusion, making it (i) harder to rebut, and (ii) a better overall contribution to the debate. The impact of analysis in a debate is especially evident when a debate goes into a metaphysical overdrive. In these situations the moral base on which the debate should be judged can be unclear, which means that the winning team can be decided based on who explained their moral grounds the best, regardless of the general persuasiveness of that morality.

4) This is an interesting problem. I would say that this depends on (i) role fulfillment and (ii) inherent persuasiveness of the direction that the team is going. To start with (i), it is in general the case that the first half has to care less about being “on clash” than the second half. The first half can partially determine where the debate takes place, and which consessions are being made. This does not mean that the second half has to follow this framework. It is however the duty of the second half teams to analyse why their particular take on the debate is also “on clash”, as they are deviating from the initial clash in the debate. This extra burden is required to ensure that the second half (or one of the teams) cannot just start a different debate without engaging in the first half debate (unless there was a squirrel, perhaps, but in that case the “rules” regarding case-drops become applicable”). So in the given example, the second opposition has to argue why their different take on the debate is legitimate. If this is not done, I would say that your analysis could fall out of the debate. Also, if the first opp already conceded that ANY referendum is a bad idea, you are not just off-clash, but you would also be stabbing the first half. (ii) inherent persuasiveness with argumentation can dictate that the specific take a team took on the debate is not the main clash in the debate, and is therefore “off clash”. For example, if the motion is THW invade Syria with tanks, mines, soldiers and dogs, it would seem that argueing from opp that dogs are a bad idea would in principal be an opp to the motion, but seems inherently off-clash to the rest of the debate.

I really value analysis on the “rules” of BP, because they allow us to endevour into the trickier areas of debates. That is why I wrote this reaction. I however believe that the Dutch circuit has plenty of good judges.

How not to judge: Part 1