Top Law Schools Boost AI Training as Legal Citation Errors Grow

Incidents of AI-generated errors in legal citations have increased the pressure on law schools to teach responsible use of the technology.

The University of Chicago, University of Pennsylvania, and Yale law schools are among those augmenting curricula. In new or updated classes, schools are training their students to understand the AI tools’ limitations and to check their work.

“You can never give enough reminders and enough instruction to people about the fact that you cannot use AI to replace human judgment, human research, human writing skills, and a human’s job to verify whether something is actually true or not,” said William Hubbard, deputy dean of University of Chicago Law School.

The elite law schools’ program changes come after attorneys have been fined or faced sanctions for their unchecked usage of artificial intelligence. The schools are updating and expanding their offerings to better prepare young lawyers for the risks generative AI poses and to let students flex tech muscles while the stakes are low.

At Chicago, “Generative AI in Legal Practice,” “Editing, Advocacy, and AI,” and “Regulation of AI: Legal and Constitutional Issues” are examples of classes offered this fall. The law school intends to enroll 15-20 students per class.

Hubbard said professors and administrators are also trying to incorporate the tool into doctrinal classes and clinical teaching. “Some professors are embracing it more aggressively than others,” he said.

Ahead of Game

Students and faculty at University of Pennsylvania Carey Law School will have access to ChatGPT Edu in their legal writing classes this fall to foster conversations about using the tool in their work. The school will enroll approximately 300 students, which includes all of the first-year students and the third-year legal writing fellows, into the OpenAI program. This “secure” model of ChatGPT, where inputs are not shared with public versions, was made specifically for universities.

“We can’t pretend (AI) isn’t happening, it’s going to happen,” said Professor Polk Wagner, deputy dean for academic affairs and innovation at Penn. “It’s going to have to be an important feature of modern legal education.”

Along with the rollout of ChatGPT Edu, the school is offering two seminar courses this fall for approximately 12 students each. These additions are teaching students to be “ahead of the game” so they can be competitive, Wagner said.

In Bloomberg Law’s Path to Practice fall 2024 survey, 36% of professors and 33% of law students said that AI has been included in their elective course offerings, while 6% of students and 12% of professors said AI has been included in doctrinal classes.

Aside from the risks that sloppy use of AI engenders, there’s also an existential concern that AI could eliminate jobs.

Major law firms operate on a pyramid model with a tranche of entry-level associates forming the base and a select group of equity partners at the very top.

Nikia Gray, the executive director of the National Association for Law Placement, said the advancement of AI is going to require entry-level associates to work quicker to catch up to the caliber of a mid-level associate.

This “senior leverage model,” as Gray describes it, will continue to emerge as large law firms invest in AI. This model, which Gray said favors experienced attorneys over junior ones, will lead to law firms decreasing entry-level hiring as AI will conceivably do the tasks that junior associates do now.

Student Driven

Students at Harvard Law School last year cited the lack of AI integration into coursework and the rapid development of the technology as the reasons they created the Harvard Law Artificial Intelligence Student Association. With about 250 subscribers, it explores the ethical, legal, and societal impacts of AI.

Kevin Wei, a co-founder and rising third year, said the group is a “big-tent organization.”

Last year Harvard’s policy for AI was updated to allow students to use AI in the same way they’d use non-AI resources. However, Wei said he doesn’t recall a professor ever explicitly allowing classroom usage and said that the updates haven’t been widely communicated to students.

Still, Wei said the policy change is a “promising sign” that administration is beginning to take AI seriously.

Harvard Law didn’t immediately respond to a request for comment.

Lab Work

Chicago last week launched a lab where students will design an AI chatbot focused on renters’ rights with Kimball Dean Parker, the CEO of legal tech company, SixFifty.

At Yale Law School, Professor Scott Shapiro teaches “Media Freedom and Information Access Clinic,” a pro-bono legal clinic for independent documentary filmmakers. To serve more clients, students train language models about media law and learn its setbacks and limitations as well as practical usage.

“About a year and a half ago, I thought, we’re always addressing the same issues, why don’t we just build an AI model,” Shapiro said.

_{Podcast: When a Law Firm’s Tech Venture Goes South, the Lawsuits Fly}

Using data collected from pro bono work, he taught a large language model about media law. After students trained the model to think like a media attorney, they fed it memos that students wrote in the clinic. The model’s outputs are then checked for hallucinations.

“At Yale we don’t just teach law students law, we teach them how to teach AI models law,” Shapiro said.

Northwestern University’s Pritzker School of Law offers a collaboration with the Computer Science Department in the McCormick School of Engineering. Interdisciplinary teams work to provide AI prototypes for external partners including Adobe, Thomson Reuters, and Allstate.

Daniel Linna, the director of law and technology initiatives, said providing the space for innovation, as well as coursework dedicated to AI, are essential.

“We’re doing a disservice to our students if we don’t let them know how to use these tools well,” Linna said.

He added, “If you’re not positioned to do that as a student when you leave law school, you’re going to have less career success long-term.”