New study suggests cutting bar exam multiple choice questions or making them open book
Image from Shutterstock.com.
Bar exams should use multiple choice questions sparingly, if at all, replace essay questions with performance tests, and require supervised clinical work for licensure, according to a study focused on minimum competency to practice law by the Institute for the Advancement of the American Legal System.
The study was released Wednesday. It also recommends making essay and multiple choice questions open book and providing test-takers more time to complete them.
Titled Building a Better Bar: The 12 Building Blocks of Minimum Competence, the study was authored by Deborah Jones Merritt, a professor at the Ohio State University’s Moritz College of Law, and Logan Cornett, director of research at the IAALS.
In 2019 and 2020, they assembled 50 focus groups comprising 200 participants. Of those 50 groups, 41 comprised new lawyers, and nine comprised people who supervised new lawyers.
According to Merritt, this is the first study to use in-depth discussions, rather than surveys, to identify skills and knowledge needed for practicing law. Also, the study claims that while the bar exam tries to distinguish minimum competence to practice law, there has never been an evidence-based, agreed-upon definition of minimum competence.
“The legal profession prides itself on its integrity. But if we are to meet our own expectations—and those of the public—we must adopt an evidence-based definition of minimum competence. We must also use that empirically grounded definition to shape the lawyer licensing system. Our research provides the critical first step on this path,” the authors wrote in the study’s executive summary.
The National Conference of Bar Examiners develops and produces bar exams used by most jurisdictions. In 2018, it appointed a testing task force, which since then has engaged in listening sessions about the exam’s content, format, timing and delivery method.
In an Oct. 29 email to the ABA Journal, the NCBE wrote that the task force will release the third phase report of its study next week. It plans to share preliminary recommendations from stakeholders in December and publish finalized recommendations in January.
“The findings of the IAALS qualitative study complement the results of NCBE’s empirically based practice analysis, particularly those findings pertaining to the knowledge and skills required for minimum competence. However, some of the IAALS report’s insights and recommendations regarding the utility of different assessment formats, their strengths and weaknesses and how they fit together into a comprehensive licensure program appear to be based primarily on opinion rather than research or best practices for licensure testing. NCBE’s testing task force will evaluate IAALS’s recommendations and consider their report as it finalizes its recommendations for the next generation of the bar exam,” according to the email.
Updated Oct. 29 at 4:27 p.m. to add the statement from the National Conference of Bar Examiners.
