California announces online bar exam, but how will it work?
Image from Shutterstock.com.
If the novel coronavirus pandemic leads to online remotely proctored bar exams, controlled test-taking environments, one of the most equalizing factors, will be eliminated.
Instead, some people will have the privilege of taking the tests from well-lit, comfortable, quiet home offices with plenty of space. Others will be forced to make do in cramped living quarters with many disturbances—a situation that could be flagged by proctors as possible cheating.
Such scenarios raise questions about whether exam results would be fair or valid, says Chad Buckendahl, a psychometrician and partner with ACS Ventures, who has done various reports for the State Bar of California.
“If you think about licensure exams that are administered through testing centers, the centers are not identical in terms of makeup, but they’re equivalent in terms of the amount of space people have and the lighting. I would be concerned about someone taking the exam in a home environment,” he adds.
And that’s just one worry about an online remotely proctored bar exam, which is in the works for California, and an option in Massachusetts if an in-person test can’t be safely administered. Other concerns include test security, whether good platforms can be built and tested in time for the exam and how states will meet Americans with Disabilities Act accommodations if a test is administered remotely.
In addition to online remotely proctored bar exams, jurisdictions are thinking about adjusting testing locations, setups, protocols and staffing to meet health guidelines, according to Judith Gundersen, president of the National Conference of Bar Examiners. The NCBE develops and produces the attorney licensing tests used by most U.S. jurisdictions.
If states using NCBE tests opt for a remotely proctored online bar exam, outside vendors would provide proctors who view test takers through webcams, Gundersen says. Making sure the exam questions are kept secure during testing and candidates aren’t cheating are significant parts of ensuring test scores are valid measures of candidate proficiency, she adds.
“An online exam that is remotely proctored would not offer the same level of uniformity of testing conditions and security as a normal bar exam administration,” she wrote in an email to the ABA Journal.
Martin Pritikin, the dean of Concord Law School at Purdue University Global, thinks there may be a good solution for test security: In addition to a front webcam recording, a second recording could be made from the back, with a smartphone far enough away to capture the test-taking environment.
“I think you could actually have it be more secure than in-person testing,” says Pritkin, whose law school is entirely online and does not have ABA accreditation.
Using remote proctors is not new in testing, but what’s unknown is whether it works well for detecting and eliminating cheating or if it can gather enough evidence to show cheating actually occurred, says James Wollack, an educational psychology professor at the University of Wisconsin-Madison.
“What happens when someone’s 6-year-old walks in during the test, and their spouse comes in running after them? Are you cancelling the test?” Wollack asks.
He also thinks meeting ADA accommodations would be difficult with a remotely proctored bar exam. Gundersen agrees, pointing out that extra testing time can be delivered, but other accommodations might require jurisdictions to administer the exam by another method.
She adds that accommodations such as taking stretch breaks or reading exam materials aloud could raise false red flags for cheating in a remotely proctored exam.
“Depending on the nature of the disability, some candidates may need a distraction-free environment and if their living situation doesn’t provide that, it could impede them from clearly demonstrating what they know,” Gunderson wrote. “Similarly, the presence of a black box artificial intelligence system may trigger anxiety for those who have generalized anxiety disorder.”
And some say testing security would be difficult if not impossible for a two-day remote test. You could have a shorter version of the exam’s multiple choice questions, Wollack says, but the traditional test reflects data the NCBE has collected about content knowledge that’s necessary for newly licensed lawyers to have, in order to protect the public.
“So shortening the tests would, to some extent, come at the expense of measuring this content domain with precision,” Wollack explains. He adds that formatting the questions and getting them into a system, as well as adequately testing your platforms, would be hard to have done by July.
“People say, ‘Just put the test online. This isn’t a situation like the [traditional] bar exam, where we’re all sitting in the same ballroom taking the test. It’s actually a technical system, and you can’t just wish it into existence; you have to build it,” says Kellye Testy, president of the Law School Admission Council. Her organization is offering its first remote LSAT in May, and another is planned for June.
This was possible, she adds, because they determined, based on decades of question data, that the test could be shortened from five to three sections and still have valid and reliable results.
Testy describes an online remotely proctored bar exam as a “very heavy lift,” and says many states are planning for the scenario but hope they can ultimately administer a traditional in-person bar exam in the summer or fall.
“I’m not going to say it would be impossible or that it won’t happen, but it’s a really big lift for states. They’re not testing organizations,” Testy says.
