By Srdjan Miletic*
Knowing how to judge is important. It’s important if you judge as you need to give fair calls and concise, useful feedback. It’s also useful when speaking because you only win by convincing the judges and knowing how they think helps. The issue with judging at the moment is that, while we have a general consensus on how debates should be judged, there remain a significant number of often unnoticed yet important differences. Differences which, if left unreconciled, will lead to inconsistent judging and unfair calls. In this article my aim is to tackle a few of the most common issues I have seen on the circuit and to propose solutions to them. Since most of the solutions come together nicely in my own judging model I’ll present that model first and then go about explaining how it resolves said issues. Note that the solutions I propose are based on a conception of what debating should be that is very much personal to me. That conception is that debating should be as rational, interesting, fair and strategically rich as possible, I realize that many of the people reading this may well have a different conception or disagree with my views on more practical grounds. Weather you agree with me or not, if you are organizing or hosting an debating tournament I urge you to take a stance on these issues and make that stance clear to both debaters and jugs attending your tournament. Only then can fairness be ensured.
Srdjan Style Judging TM
My own way of judging is by no means unique or entirely original but has been cobbled together from a variety of sources and tinkered with to solve some of the more serious issues I have had with the standard “whatever is left standing” judging model I was taught. Specifically, the movement model of judging, which I cannot for the life of me remember where I read, forms the foundation of my approach. For me debating is about arguments. A team should be judged based on their logical persuasiveness, meaning that speakers should not be explicitly given credit for rhetoric and style. A teams contribution to the debate should be measured by the sum of the persuasiveness of all the individual arguments made. This means that a team should never win because, for example, they make more arguments than their opponents as it is the total quality, not quantity, that matters. How persuasive each argument is should be judged as follows: Truth (0.0 – 1.0) * Importance (0 – 100) * Originality (0 – 100). Originality means not to how novel or stale an argument is but to what extent it was already brought by other teams and is necessary to ensure that teams do not get credit for repeating what others have already said. Rebuttal or integrated destruction of another teams arguments should be assessed the same way except in terms of how much it reduces the truth or importance of other teams argumentation. Hence, utterly destroying a strong argument should be equally persuasive as bringing said strong argument. Note also that a team should not be penalized or have credit removed if their arguments are rebutted as rebuttal is already rewarded. Finally POI’s should be treated the same way as points made in a speech. Not taking a POI should either result in the chair intervening (e.g: The EUDC compulsory POI rule) or in the assumption that a devastating POI was given and not answered, although in this one case use the weakness exposed by the POI to remove credit from the speaker instead of distributing it to opposing teams. Now that how I think judging should work is clear I’ll move on to looking at a few common disagreements and explain how they are resolved by the adoption of the above standard.
Issues solved by my judging model
1.Does Style Matter?
Style will always affect persuasiveness subconsciously but the question is whether judges should try to repress this instinct or welcome it. As far as I know there are two main arguments in favour of judging at least partially based on style. One, style makes a real life speech easier to listen to and more persuasive and this should be reflected in BP. Secondly style prevents debaters from “machine-gunning” as no style points means that a debater always has an incentive to speak as quickly as possible. This is both frustrating to judge and punishes ESL speakers who cannot achieve the same speed. I believe machine gunning can be prevented simply by ignoring material which, due to speed, would not be intelligible to the average, reasonably well informed voter. As for rhetoric being pleasing or persuasive, I agree but think BP should be about reason and argument although this view is, admittedly, a matter of personal preference.
2.Engagement
Often I hear judges explicitly reward teams for engagement. For instance, saying that a team should get a higher position because they “engaged” more. There are two issue with rewarding engagement explicitly. Firstly, it’s difficult to determine just how much a team has engaged and, in the absence of an objective standard, I fear that judges will be unduly influenced by style and reputation. Secondly, a team with stronger argumentation may lose to weaker teams due to their (stronger) arguments taking a different approach than the other teams in the debate and hence engaging less with those other arguments. For example, in a debate about torture a team may choose to focus on the political ramifications of torturing terrorists while the other teams all focus on effectiveness. In this situation the team focusing on political ramifications would likely be downgraded on engagement simply because less of their argumentation directly contradicts other argumentation revolving around how effective torture is at extracting information. My issue with this is that it is arbitrary and hence unfair. Also, I find it strange that a team which brings worse arguments can win over a team with better arguments and that team are penalized for bringing new issues into a debate.
The solution to the problems with rewarding engagement explicitly is to reward it implicitly in the same way we reward argumentation. Namely, if we give credit to speakers for creating strong arguments we should give them equal credit for destroying them. This solution is simple and an extension of something we all already do, reducing the potential for confusion and making judges life easier. It’s also based on evaluating arguments and hence should be far more objective while avoiding both of the aforementioned problems. One drawback of this approach is that it allows opening teams to delay making their most important arguments in order to minimize other teams ability to rebut those arguments. I don’t believe this is a significant issue as the drawback of backloading argumentation is that less time is spent developing and analyzing a team’s best material, leaving them in a worse position when facing second-half teams attempts to rebut/steal/outimpact arguments.
3. Material left standing vs arguments brought/destroyed
When I first started debating I was, as far as I can recall, advised to reward teams for engagement. Specifically, at the end of the debate the team whose remaining (unrebutted) arguments combined were the most persuasive should get the first. The problem with this approach is that a team with stronger argumentation may lose to a weaker team due to their opponents choosing to focus on rebutting their arguments. For example, 1st Prop may lose to 2nd Prop despite bringing stronger argumentations simply because Opp chose to focus on rebutting closings arguments, leaving less of them standing at the end. This is arbitrary, unrelated to a team’s actions and hence unfair. Also, I find it strange that a team which brings worse arguments can win over a team with better arguments.
My solution to this problem is to credit teams for taking down their opponents arguments but to not remove credit from the teams whose arguments are rebutted. This means that engagement and contribution to the debate overall are still accounted for but a team can no longer win over another despite contributing less to the debate.
4.Contradiction
Contradiction can occur within a team and between teams. Either way judges should only penalize teams for it if the contradiction in question would either be obvious to the average, resonabally well informed voter or else if it is pointed out by opposing teams The standard policy on inter-team contradiction is that closing teams cannot contradict opening teams except when opening teams make insane or obviously false claims. Assuming a closing team contradicts a reasonable claim the contradictory material is ignored. This is problematic for two reasons. Firstly, it allows opening teams to concede away arguments which could potentially form a closing teams case, thus unfairly limiting the debate. This is something I have done successfully and, with even the slightest analysis behind the concession, prevents judges from easily dismissing it. Secondly, it means that an opening team which unintentionally makes a variety of weak but sane arguments can essentially lock their closing team in to defending a strategically weak or untenable line. For a far better analysis of the problems with the current standard for knifing than I could offer and, in my opinion, the best solution refer to Shengwu’s blog:
(http://trolleyproblem.blogspot.nl/2012/02/new-standard-for-knifing.html)
His solution being:
“Closing teams that contradict a claim made by the opening team should be treated as though they had conceded entirely the opening team’s claim. The degree to which they are penalised should depend precisely on how that changes the overall persuasiveness of that teams case.”
Contradiction within a team is somewhat more difficult to deal with but for brevity’s sake, and because I don’t have a good answer at the moment, I’d suggest applying Shengwu’s standard once again.
Other Issues
5.Speaks
Speaker points are a part of debating and, despite some objections , seem set to remain so for the foreseeable future. There are two issues with the way in which speaker points are currently assigned, leading to inconsistency and hence unfairness. Firstly, judges often spend far too little time determining speaks. Secondly, speaks are highly subjective with different judges from different circuits likely having substantially different norms. While it is impossible to define or communicate an entirely objective standard for, say, what constitutes an 80, it is possible to minimize differences to some extent through a few very simple measures. Firstly, taking speaks seriously and spending some time thinking about which scores are given and why should be the norm, not the exeption. Secondly, justifying scores in reference to the speaker scale should ensure that the discussion of speaks has at least some concrete base to rest on rather than relying solely on judges past experience.
6.Counter-Propping
A counter-prop refers to the decision by an Opp team to defend something other than the status-quo. Before I walk into this minefield I want to make one thing clear. I don’t think this should be a major issue. I think that if motions are set with counter-propping in mind then counter-propping will lead to fair debates and visa versa. I think the problems only occur when CA’s and tournaments don’t make clear what their policy on counter-propping is, leading to confusion and debates which, by going against the spirit of the motion, can be drastically unfair. So, whatever your opinion, just make sure to ensure that everyone else at the tournament; CA’s, speakers and judges are on the same page and there shouldn’t be any problems.
Into the minefield. Firstly, should counter-propping be allowed or should Opp teams always defend the status-quo? I believe that counter-propping is good. Logically, things are only good or bad in comparison to other things. For instance being murdered is bad in comparison to being beaten but good in comparison to being raped and then murdered. Hence, without counter-propping debates may well revolve around Prop arguing that a policy should be implemented because it is bad in relation to a worse status-quo. This is problematic because the resulting debate can be nonsensical as everyone in it knows that the real question is about a different alternative policy. To illustrate consider the following example. Let’s say you have an island state which kills half its population every year for absolutely no reason. Let’s say the motion is TH, as the island state, would kill ¼ of its population every year. Now that is obviously better than the status quo but it would be insane to call such a policy good as an obviously better alternative, killing no one, exists. Likewise, even real motions such as “THW redistribute funds among disease charities based on the number of people affected by their disease” become nonsensical if opp is forced to defend the status-quo instead of pointing out that redistribution based on number affected as well as the severity of the effect and the potential of research to produce results would be better.
Secondly, if counter-propping is allowed should Opp have the same leeway as Prop when it comes to proposing policy? This may seem like a silly question but I’ve repeatedly seen judges downgrade Opp teams for proposing “unrealistic” policies which would be perfectly acceptable if proposed by a Prop team. My view on this is that Opp should have the same presumed power as prop, otherwise judges and teams have to make the tricky decision of what is plausible and what is not.
Thirdly, does an Opp team have to commit to defending a single alternative, whether it be the status quo or a counter-prop, or can they argue for a menu of mutually exclusive alternatives? My opinion is that Opp should be free to offer as many alternatives as they want and only be penalized insofar as their argumentation in favour of one alternative conflicts with their argumentation for another. This may seem unfair in comparison to 1st prop who has to defend only one policy but I believe that this is more than balanced out by Prop’s ability to choose the initial policy which opp teams must then respond too. Also, splitting time and attention between multiple alternatives necessarily means less time is spent strengthening each one.
Finally, can 2nd Opp Counter-prop or do they have to be consistent with their opening team much as 2nd Gov must? On one hand it could well be argued that allowing 2nd Opp to counterprop disadvantages 1st Gov and is unfair on a 2nd Gov who is in the same position as 2nd Opp but cannot create a new model and must instead extend on 1st Gov’s model which may well be suboptimal. I believe that engagement with 1st Gov is not an issue as first-half teams inability to engage is balanced by their ability to take the lion’s share of available argumentation, hence why it is also acceptable for 2nd Opp to bring entirely novel arguments which 1st Gov also cannot respond to. As for the 2nd Prop/2nd Opp imbalance I think this is a serious issue with two solutions that I can see. Either 2nd Opp should be treated the same way as 2nd Gov and be forced to defend openings case or both closing teams should be allowed to propose new models as long as those models are not contradictory with first half. Both approaches are problematic and I can’t make a decision as to which is better although, if pressed, I’d choose the first simply because it is closer to what we do now and hence is less likely to have unexpected consequences.
Conclusion
The issues I’ve covered here are by no means the only areas of judging lacking clarity and my analysis is far from perfect. More than that deciding how to judge debates means deciding what debating should be about and that’s not a decision I can make alone. Nevertheless, I hope I’ve at least shed some light and brough some discussion to issues that have so often been unnoticed. Finally, if you have any comments, questions or suggestions feel free to contact me at teddingtonster@googlemail.com
*= Coach, Rhetorica Debating Society Maastricht
Mascha Bloemer
Mascha is een alumnus van de Amsterdamse Studentendebatvereniging Bonaparte. Zij was redacteur van SevenTwenty (2012-2013) waarna ze in 2013-2015 de rol van hoofdredacteur op zich pakte.
Over de auteur