Robot Lawyers: Sooner Than You Think
By Haley Magel
There’s been a lot of buzz recently about ChatGPT and robots taking the role of lawyers[1], and many probably think it’s satire or an over-exaggeration. While robot lawyers might not be taking over the legal industry right now[2], that day might be a lot sooner than anyone expects.
For those that don’t know what ChatGPT is or have a scant understanding, it is a chatbot that uses “natural language processing to understand and respond to human communication.”[3] Chatbots are either retrieval or generative, and ChatGPT is generative meaning that it takes a user input pattern and creates the output itself with the help of an underlying deep-learning model. [4] In less technical terms, you can ask ChatGPT a question and it will answer the question with an output that it creates. In the legal context, one could ask ChatGPT to explain what constitutes a well-founded fear of persecution in an asylum case, and the chatbot can spit out a relatively accurate response.[5] One could also ask ChatGPT to explain the concept of personal jurisdiction, develop a list of deposition questions for the plaintiff in a motor vehicle accident, and create a contract for the sale of real estate in Massachusetts and receive competent answers.[6]
There are obviously some drawbacks to ChatGPT as it currently operates such as low interpretability which means that ChatGPT does not explain the methods it uses to come to its answers.[7] ChatGPT also does not include footnotes or specific references, so it isn’t easy to fact-check answers and make sure that the correct legal authority was applied accurately.[8] Another factor to take into consideration is that all artificial intelligence (“AI”) is trained with human input and there are numerous examples of how bias has been introduced into algorithms.[9] Drawing conclusions from AI could include implicit bias that many aren’t suspecting to be in the AI output.[10]
The ultimate test of legal competence for many is the bar exam, so researchers put ChatGPT to work to try its hand at answering questions from the multistate multiple choice section of the bar exam, known as the Multistate Bar Examination (“MBE”).[11] ChatGPT’s answers were compared to the average correct answers of bar test-takers, and overall bar takers answer 68% of questions correctly with ChatGPT answering 50% of questions correctly.[12] ChatGPT is significantly exceeding the baseline random choice rate of 25%, but is still trailing human test–takers by 18%.[13] Researchers believe that a chatbot may be able to pass the bar exam within the next 18 months as updated versions of ChatGPT are released.[14]
Recently, there actually was an attempt to use ChatGPT in a courtroom setting where DoNotPay tried to use its AI chatbot to help represent a defendant in a speeding case.[15] The plan was to have the chatbot run on a smartphone, listen to what was being said in court, and provide instructions to the defendant via an earpiece.[16] State bar association prosecutors threatened 6 months of jail time if a chatbot were used in court, and DoNotPay backed down with their robot lawyer stunt.[17]
Thankfully, for every lawyer out there that wants to keep their job, it doesn’t seem like ChatGPT and other AI chatbots are ready to take over the legal industry quite yet. It seems most likely in the near future that chatbots will be used in conjunction with human lawyers to achieve simple drafting tasks and other small, routine legal needs.
Image Source: https://miro.medium.com/max/900/0*zvGIPN52Cx3NVGvP.png
[1] Ken Crutchfield, ChatGPT—Are the Robots Finally Here?, ABOVE THE L. (Jan. 10, 2023, 1:47 PM), https://abovethelaw.com/2023/01/chatgpt-are-the-robots-finally-here/.
[2] Amanda Yeo, DoNotPay’s AI Lawyer Stunt Cancelled After Multiple State Bar Associations Object, MASHABLE (Jan. 26, 2023), https://mashable.com/article/donotpay-artificial-intelligence-lawyer-experiment.
[3] Thomas Bacas, Analysis: Will ChatGPT Bring AI to Law Firms? Not Anytime Soon, BLOOMBERG L. (Dec. 28, 2022, 10:22 AM), https://news.bloomberglaw.com/bloomberg-law-analysis/analysis-will-chatgpt-bring-ai-to-law-firms-not-anytime-soon.
[4] Id.
[5] Jenna Greene, Will ChatGPT Make Lawyers Obsolete? (Hint: Be Afraid), THOMSON REUTERS (Dec. 9, 2022, 2:33 PM), https://www.reuters.com/legal/transactional/will-chatgpt-make-lawyers-obsolete-hint-be-afraid-2022-12-09/.
[6] Id.
[7] Bacas, supra note 3.
[8] Crutchfield, supra note 1.
[9] Id.
[10] Id.
[11] Michael James Bommarito and Daniel Martin Katz, GPT Takes the Bar Exam, at 2, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4314839.
[12] Id. at 5.
[13] Id.
[14] Id. at 6.
[15] Yeo, supra note 2.
[16] Id.
[17] Id.