California State Bar Admits to Using AI for Exam Questions

The State Bar of California sparked outrage after it admitted to using artificial intelligence to help craft some of its multiple-choice exam questions, the Los Angeles Times reported. The Monday announcement came after test takers complained of glitches and irregularities while taking the California bar exam in February.

In a news release, the State Bar of California promised to petition the California Supreme Court to adjust test scores for those who took the exam in February. The release detailed that the test’s multiple-choice questions were mostly developed by the test company Kaplan, while some were recycled from the First-Year Law Students’ Exam and others were developed by ACS Ventures, the State Bar’s independent psychometrician, hired to assess questions. ACS Ventures used AI.

But State Bar officials defended the veracity of the exam’s questions.

“We have confidence in the validity of the MCQs to accurately and fairly assess the legal competence of test-takers,” State Bar executive director Leah Wilson said in the release. “Lessons learned are being incorporated into the July exam, and all future tests will include additional levels of independent review and validation.” 

Test takers and law school faculty have reacted with shock.

Katie Moran, an associate professor at the University of San Francisco School of Law who specializes in bar exam preparation, told the Los Angeles Times that the announcement was a “staggering admission.”

“The State Bar has admitted they employed a company to have a non-lawyer use AI to draft questions that were given on the actual bar exam,” she said. “They then paid that same company to assess and ultimately approve of the questions on the exam, including the questions the company authored.”