Democracy without Enlightenment: A Jury Theorem for Evaluative Voting

Abstract

Say a jury is going to decide who wins a competition. First, each member evaluates all the competitors by grading them; then, for each competitor, a collective grade is derived from all the judgments of all the members; finally, the jury chooses as the winner the competitor with the highest collective grade. This is <i>collective grading</i>. The grades that are used might typically be numerical scores, or evaluative expressions of a natural language, such as “good,” “fair,” and “bad.” They could be any signs at all, though, that come in a “top” to “bottom” order: thumbs up and down; happy, neutral, and sad emojis; or cheering, clapping, booing, and angry hissing at public events. Panels, boards, and committees throughout society evaluate all manner of things by grading them. Thus risks are prioritized, research proposals are funded, and candidates are shortlisted for jobs. Apart from acclamation in special cases, collective grading is not a usual way to pick winners in political elections.<p><p>

This article takes up a question about the quality of judgments and decisions made by collective grading: under which conditions are outcomes likely to be right? An answer comes in the form of a jury theorem for <i>median grading</i>. Here, the collective grade for a thing is the median of its individually assigned grades—the one in the middle, when all of them are listed from top to bottom. Section III prepares the ground for this theorem by discussing different senses in which grades can be the <i>right</i> ones for things, or the wrong ones as the case may be, independently of which grades are assigned in the end. These notions of right and wrong are relevant to judgments of different kinds of things: risks, research proposals, job candidates, options in referendums and elections. The grading‐jury theorem in Section V identifies conditions on the grading competence of individual people under which median grades, and decisions that follow them, are likely to be, independently, right.