In the last day, a political science experiment has become front page news in Montana. While its unsurprisingly being covered by InsideHigherEd, rarely is something as mundane as a randomized mailer the subject of so much political news coverage. Indeed, thousands of instances of Get-Out-The-Vote postcards, phone calls, emails, and door-to-door canvassing occur in every election cycle, with a tiny fraction of those voter contacts being overseen by social scientists. Most of those calls, mailers, and door-knocking attempts are ignored. This experiment, it seems, is different.

The Experiment

Montana Secretary of State, Linda McCulloch, has filed a formal complaint (available here), with the state’s Commissioner of Political Practices, alleging that this experiment violated several campaign and elections laws. Specifically, the experiment which appears to have been conducted by Stanford’s Adam Bonica and Dartmouth’s Kyle Dropp, distributed mailers to an unspecified number of Montana residents reporting estimated ideological positions of candidates in the state’s nonpartisan Supreme Court election. (A copy of the mailer is included in McCulloch’s complaint.)

The ideological positions are drawn from Stanford’s Database on Ideology, Money in Politics, and Elections (DIME), created by Bonica and collaborators. Such estimation of ideological positions has been a feature of political science research for decades. Originally using interest group ratings to evaluate the positions of members of Congress, this methodologically rigorous subfield has grown substantially in recent years to use voting, candidate positioning, political texts, campaign contributions, and other data to map the ideologies of state legislators, U.S. Supreme Court justices, members of Congress, among others. Such research is valuable because it largely shows that American politics is reducible to a dominant left-right political divide and that we can learn a considerable about the positions of politicians, voters, judges, media outlets, and other actors from what they do and say, and who they interact with politically. Given that there is generally little information about individual candidates and politicians (aside from party membership and endorsements), such information can also be valuable to citizens, voters, and media for making sense of the positions of their elected officials for the purposes of arriving at voting choices and deciding where to expend time, money, and other political resources.

The particular details of the Montana experiment are not known, but a quick look at the mail piece and talking points released by Stanford University suggest that the study had a relatively commonplace and perhaps innocuous intent: to understand whether information provision increased turnout in low-turnout elections. Turnout in nonpartisan judicial elections tend be quite low as voters “roll-off” the ballot before reaching lower profile elections, which are printed lower on the ballot that State executive or Federal elections. (In my home state of Minnesota, such elections are actually listed on the back of the ballot.)

In the 2012 General Election for Montana Supreme Court justice #5, 419,596 voted for a Supreme Court candidate. By contrast, 483,932 voted for a Presidential candidate. Just over 13% of voters who turned out (who were 72% of those eligible to vote) skipped voting for a Supreme Court candidate despite going to the polls. In the previous election (in 2010), 313,506 citizens voted for a Supreme Court candidate out of 367,096 that went to the polls (a roll-off of about 15%, on top of the 44% of eligible citizens who stayed home entirely). In short, even among those motivated enough to vote at all, judicial elections like the one being studied in the Montana experiment are subject to objectively low turnout and considerably lower turnout than other elections.

Backfire

Reactions to the experiment in Montana seem to have been less than positive (see here, here, here, among others). The complaints are numerous. SoS McCulloch lists four specific potential violations of state elections law: “Coercion or undue influence of voters”, “Incorrect election procedures information”, required registration of committees engaged in political campaigning, and “impersonation of public servant” due to the mailer’s use of the Great State Seal of the State of Montana.

I think a close read of the mailer and the relevant legislation will reveal that the first three of those accusations are not substantiated. Information provision cannot be construed as coercion, unless all campaign is illegal, which it is not. The mailer conveyed information about judicial candidates’ estimated ideologies but nothing about procedures for voting. The mailer did not endorse any particular candidate and appears to list the names of all candidates with equal prominence. The last issue, that of the use of the State Seal, is perhaps more problematic. Though the image itself is in the public domain, no logo can be used as an implicit endorsement. The mailer clearly represented itself as a research project sponsored by Stanford and Dartmouth, but one could imagine some voters being confused by the use of the seal. I’m not a lawyer and haven’t read all relevant state laws, but it appears that while the use of the State Seal is de facto forbidden in campaigns, it is not actually an unlawful offense. In other states where candidate campaigns have used a State Seal, the materials have simply been destroyed or ads been taken off the air following a slap on the wrist.

Political Science

The larger issue, which seems to spark outrage, relates to political scientists’ involvement in a nonpartisan judicial campaign. Less salient (at least at this point) are the fact that both involved universities are out-of-state and the research appears to have only been vetted by the Dartmouth (but Stanford) institutional review board (IRB). Menlo College professor Melissa Michelson has written about some of the ethical questions raised by the experiment and the implications thereof for the legitimacy of political science. Her blog post is worth a read as it seems to be capturing the sentiments of many political scientists.

This is an interesting reaction from my view. Looking at the mailers and inferring the study intent from the available information, this experiment, its ambitions, and its protocol appear mundane. Indeed, they look and feel completely consistent with the day-to-day life of modern political campaigns. That the mailer implied partisan endorsements by mapping candidates estimated ideological positions onto a left-right spectrum anchored by Barack Obama and Mitt Romney may raise concerns about introducing partisan information into a nonpartisan race, but I think that issue requires further discussion before jumping to conclusions.

First, nonpartisan elections do not obtain the democratic benefits that their advocates hope for. Research by Brian Schaffner, Matthew Streb, and Gerald Wright has shown that nonpartisan elections have lower turnout and lead voters to rely more on incumbency when deciding who to vote for (gated, ungated). Rather that stimulate beneficial democratic dialog, removing partisan information from elections seems to be associated with less participation and a bias favoring status quo officials. Furthermore, work by James Gibson of Washington University has shown that citizens acknowledge that the judiciary is politicized and that legal decisions are necessarily policymaking processes. Nonpartisan elections are not a holy grail of democracy and we should not treat them as sacred ground.

Second, judicial elections are not necessarily a democratic good, at least to the extent that they harm judicial legitimacy. Campaigning for judicial office involves tactics that make people somewhat less supportive of the judiciary. Particularly when judicial elections involve large and questionable campaign contributions, like in Caperton v. Massey, legitimacy is harmed by the requirement that members of the judiciary raise funds needed to campaign for office. Only 15 states use nonpartisan elections for their state Supreme Courts, with seven using fully partisan elections, and the remainder using some kind appointment process. Concerns that this experiment may have damaged the sanctity of an election process are likely legally flawed. The United States Supreme Court already declined certiorari for Sanders Count v. Bullock, a case (in the spirit of Citizens United) that declared a state ban on endorsements and outside spending in precisely these kinds of elections unconstitutional.

Finally, political scientists are presently concerned about their own legitimacy. The American Political Science Association has been on the defensive since Senator Tom Coburn thrashed the discipline for irrelevance and wasting taxpayer dollars in 2013.

Yet this experiment highlights the challenges of being relevant. Seemingly, Bonica and his collaborator(s) aimed to reach new insights into campaigns and voters behavior by supplying information in an otherwise low-information election. The evidence cited above shows that nonpartisan elections have low turnout, but there is little research showing what if anything might ameliorate that deficit. Political scientists generally agree that full, diffuse, and equal participation on the part of citizens an unquestionable democratic good. This experiment, presumably, was aimed at reaching new insights into how to improve democratic well-being and now it is being criticized for doing just that.

This is unfortunate because the media coverage now means that this experiment is an analytic lost cause. I can only imagine exactly how the researchers distributed the mailers to citizens, but presumably the aim was to compare turnout in similar precincts that were randomly assigned to receive the mailer or to receive nothing. Given the contamination of the those control group precincts, it is unlikely that this experiment will tell us much of anything relevant about the nature of American elections. That is unfortunate.

Ethics

Political scientists are right to be concerned about public perceptions of the discipline. It is also part of the political scientist’s code of ethics to conduct responsible research. Stanford and Dartmouth no doubt require their researchers to submit planned studies to IRBs; based on the content, I suspect this project was exempted from full review by Dartmouth and would have been similarly exempted by Stanford had the researchers sought review there. In short, there are problems with this experiment (the researchers probably would have been wise to not use the State Seal) but do not seem to have committed any crimes or violated any norms of ethical scientific behavior.

When we evaluate the ethics of research studies, The Belmont Report requires that we - as scientists - act according to respect for persons, beneficence, and justice. In short, research should do good and treat participants in research with care, respect, and dignity. When harm is required for research, the collective costs of that harm to particular participants must be weighed against societal benefits. The harms in this study seem limited. Individuals were provided with information that might help them vote. If we believe some of the research cited above, the harms are that this particular mailer (by adding a small amount of partisan information to an officially nonpartisan but de jure politicized election) might make them lose a little bit of their support for their state’s judicial system. The benefits, however, could be substantial: they could have spoken to potential strategies for broadening voter turnout and improving voters’ decision making.

While political science in general and these researchers and institutions in particular might have damaged reputations due to the coverage of this controversy, the experiment itself is hardly harmful and its benefits to its participants and to society almost certainly outweigh those potential harms. Political scientists are rightly concerned that the blowback surrounding this experiment will hurt their ability to conduct future research. There is no reason why this should occur. Political scientists need to present a unified voice that randomized experimentation involving beneficial treatments carries societal benefits that are worth whatever minimal discomfort they might immediately produce. Worse than this experiment would be elections where voters received no information and voted blindly, or where the only information they received was intentionally biased to portray candidates in favorable or unfavorable ways. It is also possible that voters could be better served by a judiciary that is appointed rather than elected. My intuition is that the kind of information communicated here is better for democracy than what voters already get, but now we will never know.

---