Stratified Random Sampling, Aristotle, and Democracy
What if politicians were randomly chosen from the citizenry instead of elected? The Economist recently wrote about an experiment in the German-speaking community of Belgium to choose a Citizens’ Council by lot. Other randomly selected Citizens’ Councils have been assembled in Ireland, Canada, and the UK.
Figure 1. Kleroterion in the Ancient Agora Museum (Athens). Public domain image from Wikimedia Commons
The selection for modern Citizens’ Councils is typically done in two stages. First, a mailing is sent to a relatively large randomly selected sample of citizens; for the UK Climate Assembly chosen in November 2019, 30,000 letters were sent out. In the second stage, a “representative” subsample (usually between 20 and 150 persons) is selected from the set of those who respond to the first-stage mailing. The subsample, which becomes the Council, is selected in such a way that it has the same percentage of male/female members, members from different race/ethnicity groups, members from different subregions, members from different socioeconomic classes or education levels, etc., as the region being represented.
Choosing representative bodies by lot is not new. Aristotle wrote: “the appointment of magistrates by lot is thought to be democratical, and the election of them oligarchical; democratical again when there is no property qualification, oligarchical when there is.” In ancient Athens, representatives for the Boule (Council of 500) were chosen not by election but by sortition—members were chosen randomly from a pool of eligible citizens. Juries (such as the one that condemned Socrates) and other city officials were also chosen by lot.
Figure 2. Reconstructed kleroteria from the Ure Museum of Greek Archaeology at the University of Reading. Public domain image from Wikimedia Commons
A kleroterion such as that shown (in fragmentary form) in Figure 1 was used to perform the random selection; Figure 2 shows a modern reconstruction. A kleroterion was approximately 6 feet tall and contained a grid of slots. A tube on the left side ran the length of the stone slab.
After the democratic reforms of Cleisthenes in 508 BCE, each citizen of Athens belonged to exactly one of ten tribes. The tribes were further divided into demes. A citizen had a bronze ID ticket containing his name, his father’s name, and the name of his deme.
For selecting the Boule or a jury, the following procedure was used. The ID tickets for citizens from a tribe who were eligible and willing to serve were placed in a basket corresponding to that tribe. The presiding archon (magistrate) shook each basket and drew one ID ticket from each; this person became the ticket-inserter for the tribe. The ticket-inserter drew tickets one-by-one from the tribe’s basket and placed them in the kleroterion column corresponding to the tribe, starting with the slots at the top. At the end of this process, the first column contained the ID tickets of citizens from tribe 1, the second column contained the ID tickets of citizens from tribe 2, etc.
The columns would have different numbers of tickets because some tribes had more citizens willing to serve on that day than others. But the goal was to have a council or jury with an equal number of members from each tribe. To accomplish this, the archon inserted white and black marbles into the tube on the side of the kleroterion (presumably some stirring was done). For a kleroterion with 10 columns, the number of white marbles equaled the desired jury size divided by 10, and the total number of marbles (white and black) equaled the number of ID tickets in the shortest column of the kleroterion. If a jury of 40 men was desired and the shortest column had 20 tickets, there would be 4 white marbles and 16 black marbles.
The marbles were then drawn from the tube one by one. If the first marble drawn was white, then all ten citizens in the first row were selected; if it was black, all of the citizens in the first row were excused from service. Similarly, the ten citizens in the second row were selected if the second marble was white and excused if it was black, and so on until the prescribed number of jurors was selected.
If the ID tickets were placed in the slots in random order, the Boule or the jury selected was a stratified random sample of the eligible and willing citizens, with the tribes forming the strata. The probability that a particular willing-to-serve man would be selected was equal to the number of white marbles divided by the number of ID tickets in the kleroterion column corresponding to the man’s tribe.
The Athenians put in a lot of safeguards to ensure a fair selection process. The procedure was conducted in public. The ticket-inserters were randomly selected on the day the jury was assembled, making it difficult for someone to fix the jury in advance. Of course, an individual ticket-inserter who did not want Euclid on the jury could palm Euclid’s ID ticket and insert it at the bottom of the column where, unless this was the shortest column, it could not be selected. But if the ticket-inserters stirred the baskets, thereby randomizing the order of the ID tickets, each person present from a particular tribe had an equal chance of being selected, even though the selection probabilities might vary across the tribes. The kleroterion procedure thus resulted in a representative sample of the men whose ID tickets were placed in the baskets.
Did the kleroterion procedure give a representative sample of Athens? Well, no. Citizenship in Athens was restricted to men who had been born free in Athens; it excluded women, foreigners living in Athens, and slaves. And the citizens had to have sufficient leisure to spend their days hanging around the Agora, which might have lessened the participation of poorer citizens. (Many, however, had ample leisure because women ran the households and slaves did most of the other work.)
If the tribes were of approximately equal size and if the citizens who showed up were representative of the tribes, sortition would have negated any effects of gerrymandering in assigning citizens to tribes. Gerrymandering “works” to unfairly advantage some groups in U.S. politics because representatives are elected. If, within a state with ten Congressional districts, 60 percent of the voters support party A and 40 percent support party B, party B can still win the majority of seats if the districts are drawn to achieve that goal. Suppose each district has 100,000 voters. Simply specify that each of districts 1 through 7 have 50,001 voters from party B and 49,999 voters from party A. Then party B wins the election in those 7 districts, using a total of 350,007 of the party B voters. Distribute the remaining 49,993 Party B voters approximately equally among districts 8 through 10. Party A wins each of districts 8 through 10 by huge margins, but it wins only those 3 districts. The gerrymandering dilutes the power of the Party A voters by packing them together.
Under sortition, one would expect that on average, 6 of the seats would to go to party A members and 4 to party B members even if the districts are gerrymandered. This is because each individual within a district has an equal chance of being selected to represent that district; there is no dilution of voting power. Each of districts 1 through 7 has a 50.001 percent chance of being represented by a party B member and a 49.999 percent chance of being represented by a party A member; similarly, each of districts 8 through 10 has an approximately 83 percent chance of being represented by a party A member. The average result, over all possible random selections, is 6 party A members and 4 party B members, even though a particular drawing may end up with a different allocation.
But samples selected by modern-day sortition procedures can be unrepresentative too. Just as in ancient Athens, one can restrict the pool of citizens by placing restrictions on eligibility or by making it difficult for poorer citizens to serve (for example, by not offering remuneration or travel reimbursement). Citizens’ councils of 2019 may have the same demographic makeup as the population, but that does not mean they resemble the population with respect to other characteristics.
If the members of a Citizens’ Council in 2019 were indeed a stratified random sample from the population, where every individual has the same chance of being included on the Council, then they likely would reflect the views and characteristics of that population. Sometimes one draws an unrepresentative sample by chance, but it’s rare to have a sample that’s really far off. And good use of stratification makes the chance of getting an unrepresentative sample even smaller. But the key to a representative sample with known statistical properties is complete random, rather than self, selection, and that does not happen under most contemporary sortition procedures.* Relatively few citizens respond to the initial mailing (the Sortition Foundation estimates that response rates to the first stage are typically between 4 and 7 percent). If the citizens within each demographic group who respond to the solicitation to serve on the UK Climate Assembly are, say, more likely to support a carbon tax than citizens who decline to participate, then the Assembly as a whole will not represent the public on that issue.
Still, the second-stage selection process does at least ensure that women, minorities, and lower-income persons are included on the council in proportion to their representation in the population. This is in contrast to the 116th U.S. Congress, where the record-setting 130 female members comprise just 24 percent of the total membership (up from 19 percent in the 115th Congress).
In recent years sortition has been used largely for advisory bodies or to propose solutions for later referendum votes, not for legislatures that appropriate funds and enact laws. How do they perform relative to elected bodies? The (randomly selected) jury is still out.
Copyright (c) 2019 Sharon L. Lohr
Footnotes
*It appears that many of the sortition procedures use balanced sampling, not stratified random sampling, at the second stage of selection. As used in sortition, the goal of balanced sampling is to have a sample with the same percentage of men, women, persons age 18-29, persons of color, persons in each region, etc. as in the census. But these categories don't have to be mutually exclusive (if they are, the balanced sample is also a stratified sample). The balanced sample agrees with the population for each category separately (sex, age, race, region, etc.) but not necessarily on the cross-classification. Balanced sampling can be done either randomly or nonrandomly; random balanced sampling is better because it prevents the sampler from deliberately choosing persons for the council at the second stage.
Why mail invitations to 30,000 households, when only 100 or so persons will be on the Council? The large initial sample is needed because relatively few persons respond to the mailed invitation, and some demographic groups are more likely to respond than others. If the response rate is 5 percent, the 30,000 mailed invitations yield a pool of about 1,500 persons for the second stage of selection. But that pool may contain few persons in some demographic groups—for example, it may contain many retired persons but relatively few young persons. A large pool is needed to allow selecting a final subsample with the same proportions in each demographic category as found in the population. Sometimes exact matching is not possible because no one in a particular demographic category responds to the mailed invitation.
References
Malleson, Tom (2018). Should Democracy Work through Elections or Sortition? Politics and Society, 48(3), 401-417. This article is one of nine in the journal’s special issue on “Legislature by Lot: Transformative Designs for Deliberative Governance.”
Orlandini, Alessandro (2018). Simulation of the Allotment of Dikastai. Orlandini describes the procedure used to select jurors by lot with the kleroterion. He has also posted a video with a reenactment of the procedure.
Van Reybrouck, David (2016). Against Elections: The Case for Democracy. Translated by Liz Waters. New York: Seven Stories Press. Van Reybrouck reviews history and recent examples of sortition. He is definitely a supporter of sortition!