ChatGPT goes to University of Minnesota law school and passes final exams

ST. PAUL, Minn. (FOX 9) - The artificial intelligence application ChatGPT went to law school and passed the final exams. 

ChatGPT attended law school at the University of Minnesota and passed all four final exams it took, but it didn’t ace them. The chatbot finished behind all or most of the humans who took the same tests, but it did well enough for professors to wonder whether an improved version might displace some future lawyers.

The ChatGPT impressed its four professors at the University of Minnesota Law School even while earning just a C+ average on its final exams.

"What it did was pretty remarkable and well above and beyond what I would’ve anticipated four or five months ago," said University of Minnesota Law School professor Daniel Schwarcz, one of the four professor experimenters along with Jonathan Choi, Kristin Hickman, and Amy Monahan.

The artificial intelligence may not have been top of the class, but it’s gotten a lot better quickly. So maybe uploading all the legal volumes from the country’s seventh biggest law library could make it a better student. Or maybe not.

"If you fine-tuned it by feeding it a huge corpus of legal documents, I don’t think that would be enough," said associate professor Choi. "What you would need is for it to be able to compare ‘This is a bad legal document. This is a good legal document.’"

Choi says ChatGPT struggles to identify legal issues, and it can’t do deep legal analysis. But artificial intelligence is already doing simple legal work, like disputing parking tickets, on websites like DoNotPay.com.

The AI could help low and middle-income families access legal help for things like wills, end-of-life care directives, and simple divorces.

"You can really see AI being revolutionary both in making those services available, making them cheaper, and potentially taking jobs," Schwarcz said.

The Minnesota Law School professors say ChatGPT might be good for first drafts of legal memos and boilerplate documents, but it’s not ready for a solo practice.

"There’s still plenty of room for human lawyers," Choi said. "It’s just a matter of how ChatGPT will complement them rather than replacing them."

And ChatGPT seems to agree. FOX 9 asked it how soon artificial intelligence will replace lawyers, and it told us it’s likely AI will continue to augment the work of lawyers and make them more efficient, but full replacement is unlikely anytime soon.

Surely a relief to some lawyers.