The Honorable ChatGPT: How AI Systems Could Alter and Perhaps Improve the Judiciary
By Kevin Frazier*
“Judges are like umpires. Umpires don’t make the rules, they apply them. The role of an umpire and a judge is critical. They make sure everybody plays by the rules, but it is a limited role. Nobody ever went to a ball game to see the umpire.”[1]
Chief Justice John Roberts made this oft-quoted remark during his confirmation hearing back in 2005.[2] Perhaps he didn’t realize that less than twenty years later, umpires may have one of their most important roles — calling balls and strikes — replaced by an automated system. As reported by the Verge, the MLB may replace umpires with an “AI-powered system that calls every pitch a ball or strike” as soon as the 2024 season.[3]
If umps and judges really do merely apply the rules, then is automation destined to disrupt the judiciary as well? From the vantage point of 2005 or even late 2022, the meaningful replacement of human judges with automated systems may have seemed impossible or certainly unlikely in the near future. However, advances in text-based automation, as evidenced by ChatGPT, indicate that AI systems may soon possess the capacity to interpret and apply the law, in addition to balls and strikes.[4]
Before diving into whether AI systems could actually write coherent opinions, it’s important to state why automated adjudication is worthy of consideration. Impartiality and independence are fundamental aspects of a democratic system of justice.[5] In recognition of the inevitability of bias among humans, our founding fathers instituted a framework intended to mitigate external influences on judges and to afford judges the insularity necessary to lean as much on objective reason as possible when deciding cases.[6]
This framework would serve no purpose if humans could simply will themselves into complete neutrality. However, given the natural predisposition of certain judges towards certain parties and causes,[7] checks on internal and external pressures not only reduce the odds of biased decision making but also increase the odds of the public perceiving judges as impartial, independent, and fair.[8] Unfortunately, these checks have not evolved in response to new threats to the maintenance of the public’s perception of a neutral judiciary.
Judicial elections for state court judges have become a greater and greater threat to judicial bias.[9] States originally adopted in the mid-1800s as a means to reduce partisan powers controlling judicial selection.[10] However, these elections now exacerbate the influence of parties and special interests — following US Supreme Court opinions that allowed candidates greater freedom with respect to indicting their jurisprudence[11] and that opened the door to dark money in judicial contests,[12] these once sleepy elections have become extensions of legislative races and ideological squabbles. Consider that, according to analysis by the Brennan Center, $97 million was spent on state justice elections in the 2019-2020 election cycle.[13] Judges are aware of this transformation — their decisions indicate that they weigh the preferences of their supporters in addition to the facts and law at issue.[14] The change is also not lost on the public — they increasingly doubt the impartiality and independence of state courts. As the New York Times editorial board noted back in 2010, increasingly competitive judicial elections have placed the “appearance of neutrality . . . under severe threat[.]'”[15]
Concerns about external influences on federal judges have also grown and gone unchecked. Even at the US Supreme Court, special interests have found ways to gain unparalleled access to justices.[16] Federal judges on lower courts have similarly been exposed to new pressures due to worsening partisanship. Any judge aspiring to a higher court can increase the odds of being selected by partisan influences by aligning themselves with groups such as the Federalist Society—groups that espouse certain values and promote judges that advance those values.[17]
New checks to counter these pressures have not been implemented. Public financing for judicial elections would significantly reduce the need for judges to pander to donors,[18] yet this check has not gained traction. Self-regulation mechanisms among judges have similarly been ineffective — most judges retain the ability to decide whether or not to recuse themselves from a certain case.[19] And, practices that shield judges from acting as super-legislators, such as authoring anonymous opinions, are applied only in certain circumstances.[20]
All of these pressures reduce the odds of judges neutrally applying the law, adhering to precedent, predictably resolving even complex matters, and providing the public with a clear understanding of what the law is.
Automated systems carry the potential to right many of these wrongs. Let’s imagine a relatively narrow application of AI systems in a judicial context — providing a system like ChatGPT with a first stab at authoring the opinions of the Wisconsin Supreme Court; a supermajority vote by the Court could overturn any automated decision.
The AI system would be immune to many, if not all, of the pressure facing those justices. It would be impartial and independent by virtue of facing no sort of election or review by partisan officials (such is the case in courts with merit plans or appointment systems). For example, it would feel no pressure to become more punitive in election years with the goal of appeasing “law and order voters.”[21] Additionally, it would have no initial favoritism of in-state plaintiffs suing out-of-state defendants—a common bias detected along elected state court judges.[22]
The AI system would be less likely to drastically alter precedent in light of the “animalistic spirits” of voters. By virtue of being trained on the previous decisions of the court, its most likely resolution of any case would be to hew as closely as possible to prior decisions rather than write new law. To the extent that such a system would have made opinions similar to Brown v. Board of Education improbable, the court’s override authority would ensure the possibility of so called “pathbreaking” opinions, in which precedent is tossed aside, in the right circumstances.
And, the AI system would produce clearer and more easily accessible opinions. Judges do not have uniform writing styles and often feel the need to write concurrences and dissents that muddle whatever the majority opinion attempts to identify as “the law.” While these separate opinions offer the benefits of signaling possible future interpretations by the court, they come at the greater cost of confusing the public as well as litigants. An AI system could write shorter opinions in a consistent style and with simpler language than any state supreme court.
Automation has already influenced aspects of judicial decision making — though not without controversy. For instance, algorithms used to determine the flight risk of the accused have been applied by judges around the country to mixed reviews.[23] Their use serves as an illustration of the fact that once the people possess enough doubt in the impartiality, independence, and fairness of a judge, they will demand new checks to preserve the most important aspects of the rule of law.
The MLB is not set on using automation to replace umps, but it deserves credit for exploring the possibility in response to concerns about whether the rules are being neutrally and accurately applied. More and more Americans are now doubting the judiciary as a neutral arbiter — it follows that means to address and counter judicial biases ought to be considered, including automated opinion writing. Admittedly, I am unsure of when or even if AI systems capable of writing such opinions will become available. However, I am confident that the need for new checks on judges susceptible to political pressures will only grow. The law has always been malleable in the hands of human judges, but sufficient checks existed to maintain the perception of impartiality. If new checks are not created, that perception will fade and endanger the legitimacy of the entire judicial branch.
* Kevin Frazier is a clerk on the Montana Supreme Court. He graduated from the UC Berkeley School of Law and Harvard Kennedy School. His views are his own.
Image Source: https://www.mlb.com/news/electronic-strike-zone-would-be-a-game-changer/c-158512610
[1] Confirmation Hearing on the Nomination of John G. Roberts, Jr. to be Chief Justice of the United States: Hearings before the S. Comm. on the Judiciary, 109 Cong. U.S. 55-56 (2005) (Statement of John G. Roberts).
[2] See, e.g., Theodore A. McKee, Judges as Umpires, 35 Hofstra L. Rev. 1709 (2007).
[3] David Pierce, Robot umpires could be coming to Major League Baseball in 2024, Verge (Jun. 30, 2022), https://www.theverge.com/2022/6/30/23189572/robot-umpires-major-league-baseball-2024
[4] See, e.g., Ethan Mollick, ChatGPT Is a Tipping Point for AI, Harvard Business Review (Dec. 14, 2022), https://hbr.org/2022/12/chatgpt-is-a-tipping-point-for-ai (describing ChatGPT as having “crossed a threshold” with respect to usefulness).
[5] See The Federalist No. 78 (Alexander Hamilton) (stressing the importance of the “stead, upright, and impartial administration of the laws.”)
[6] See id. (discussing the structural checks on external and internal pressures facing members of the judiciary).
[7] See Melissa L. Breger, Making the Invisible Visible: Exploring Implicit Bias, Judicial Diversity, and the Bench Trial, 53 U. Rich. L. Rev. 1039, 1041-42 (2019) (“[E]ven if judges attempt to shield their decisions from their explicit biases, implicit biases may seep into judicial decision making.”)
[8] See Neal Devins & Nicole Mansker, Public Opinion and State Supreme Courts, 13 J. of Con. Law 455, 469-71 (discussing how the process of selecting judges, such as through partisan elections, impacts the public’s perception of those judges as well as how those judges approach cases).
[9] See id. at 465-67 (detailing an uptick in the competitiveness of and occurrence of “mudslinging” in judicial elections).
[10] Caleb Nelson, A Re-Evaluation of Scholarly Explanations for the Rise of the Elective Judiciary in Antebellum America, 37 Am. J. Legal Hist. 190, 205 (1993).
[11] Republican Party v. White, 536 U.S. 765 (2002).
[12] Citizens United v. FEC, 558 U.S. 310, 460 (2010) (Stevens, J., dissenting) (noting that the majority refrained from limiting its holding to elections for executive and legislative offices).
[13] Michael Waldman, Money Pours Into State Judicial Elections, Brennan Center (Jan. 25, 2022), https://www.brennancenter.org/our-work/analysis-opinion/money-pours-state-judicial-elections
[14] See, e.g., Michael S. Kang & Joanna M. Shephard, The Partisan Price of Justice: An Empirical Analysis of Campaign Contributions and Judicial Elections, 86 N.Y.U.L. Rev. 69, 72 (exploring the effect of campaign donations on judicial decision making).
[15] Editorial Board, From Scandal to Example in West Virginia, N.Y. Times (Mar. 18, 2010), https://www.nytimes.com/2010/03/19/opinion/19fri3.html?scp=1&sq=west%20virginia%20and%20public%20financing&st=cse
[16] See, e.g., Alicia Bannon, The Real Supreme Court New Isn’t the Alleged Alito Leak, Brennan Center (Nov. 22, 2022), https://www.brennancenter.org/our-work/analysis-opinion/real-supreme-court-news-isnt-alleged-alito-leak (discussing “[r]elevations about an influence campaign by anti-abortion activists and donors” on members of the U.S. Supreme Court).
[17] See Dylan Matthews & Byrd Pinkerton, The incredible influence of the Federalist Society, explained, Vox (Jun. 3, 2019), https://www.vox.com/future-perfect/2019/6/3/18632438/federalist-society-leonard-leo-brett-kavanaugh (covering examples of aspirants of judicial offices closely aligning with the Federalist Society).
[18] See Kang & Shephard, supra note 14, at 126-27 n. 213 (“f course, public financing of campaigns limits the need for private financing and therefore has the potential, depending on the design, to limit the influence of campaign contributions as well.”)
[19] Michael R. Dimino, Pay No Attention to That Man Behind the Robe: Judicial Elections, the First Amendment, and Judges as Politicians, 21 Yale L. & Pol’y Rev. 301, 338-42 (2003) (outlining the limited situations in which a judge must recuse themselves).
[20] See, e.g., Per Curiam Opinions in the Sixth Circuit, Squire Patton Boggs (July 17, 2015), https://www.sixthcircuitappellateblog.com/news-and-analysis/per-curiam-opinions-in-the-sixth-circuit/ (reporting that just 14 percent of the Sixth Circuit’s opinions were issued per curiam).
[21] See Kate Berry, How Judicial Elections Impact Criminal Cases, Brennan Center (Dec. 2, 2015), https://www.brennancenter.org/our-work/research-reports/how-judicial-elections-impact-criminal-cases
[22] See Daniel Klerman, Bias in Choice of Law: New Empirical and Experimental Evidence, J. of Inst. And Theo. Econ. [forthcoming] (2022), Available at SSRN: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4206553
[23] Megan T. Stevenson and Christopher Slobogin, Algorithmic Risk Assessments and the Double-Edged Sword of Youth, 96 Wash. U. L. REV. 681 (2018).