It seems hardly a day goes by without a story about an attorney citing hallucinated cases. Public humiliation—including court orders with scoldings and fines—is often part of the details, as well as getting fired. But perhaps the worst part is being known as the lawyer who submitted fake cases and didn’t bother to check their work.
Positive stories about how helpful artificial intelligence can be for lawyers and how much money it can make for the tech industry also take up a lot of space in the news and social media. What’s been missing from the AI buzz until recently is what attorney regulation agencies are doing in regards to the technology, and that may be because many lawyers who prosecute and defend conduct cases say no changes are needed.
“What we have in place is sufficient. They’re the same issues that lawyers will encounter using other technology,” says Seana Willing, the State Bar of Texas chief disciplinary counsel. “And until we learn otherwise, it’s going to cover whatever comes about from a lawyer’s unethical use of AI.”
Lucian Pera, an Adams & Reese partner who was a member of the ABA Ethics 2000 Commission, says existing Model Rules simply need to be interpreted against the new technology and the way lawyers are using it.
“The consensus among ethics people—and I share it—is that the rules are just fine, thank you very much,” says the Memphis, Tennessee, lawyer, who thinks the technology can be a good tool for lawyers and clients.
“Now it’s got to be used right, but that’s true of every technology that we’ve seen added to lawyers’ lives in the last 25 years,” Pera adds.
Nonetheless, some states are considering action, including California. The State Bar of California is trying to address AI directly with specific proposed changes to the California Rules of Professional Conduct. A proposed comment to Rule 3.3, which addresses candor toward the tribunal, would expressly require lawyers “to verify the accuracy and existence of cited authorities.” This means they must ensure that no cited authority is “fabricated, misstated or taken out of context, including any cited authorities generated or assisted by artificial intelligence or other technological tools.”
An additional proposed comment addresses Rule 1.4, which focuses on client communications, and may compel lawyers to evaluate how confidential information is processed through third-party systems.
Meanwhile, another suggested change to Rule 1.6, which focuses on lawyer-client confidentiality, defines “reveal” to include exposing confidential client information to AI tools.
Also, in June, the Rhode Island Supreme Court amended its rules to make clear that lawyers bear responsibility for the accuracy of AI-generated work and to clarify that client confidentiality obligations apply to data shared with AI systems.
And effective June 15, the Florida Supreme Court, in its Rules of General Practice and Judicial Administration, expressly authorizes judges to impose sanctions on attorneys and pro se litigants for submitting filings with fabricated or inaccurately cited authorities, including AI-generated citations.
Additionally, the Ohio Board of Professional Conduct, at the direction of the state supreme court, recently published an ethics guide regarding AI for lawyers and judges. The guidance offered is nonbinding.
“We’re not supposed to have to rewrite the rules every time we have a new piece of technology,” says Ryan Groff, a lecturer on legal technology at New England Law Boston.
He notes that the ABA’s 2024 Formal Opinion 512, which is about lawyer use of generative AI tools, relied on an existing ethical framework, since lawyers already had to understand the benefits and risks associated with technologies used to bring legal services to clients.
But lawyers often want more regulation, says Jonah Perlin, a Georgetown University Law Center professor whose scholarship focuses on the intersection of legal ethics, technology and legal communication.
“The idea is if you create more regulation, you tell people what they can’t do, but you also implicitly or explicitly are telling them what they can do,” he says. “Lawyers want answers because they want certainty.”
Regarding the proposed changes in California, however, Perlin thinks the proposed comment to Rule 1.6 regarding AI and client confidentiality may be overly protective.
According to Perlin, the current language could chill lawyers’ use of AI by creating the impression that the use of most AI tools creates “a substantial risk.”
Speaking more generally, he adds, “I’m a big believer in creating rules that tell lawyers the process they need to go through and what they need to care about. But ultimately, leave it to those lawyers and their clients to make risk-based decisions about what they can and can’t do.”
Jean Cha, a former State Bar of California prosecutor who now defends lawyers in legal ethics matters, says she has seen a “huge uptick” in discipline cases involving AI misuse. But she also thinks California’s proposed changes may go too far.
She asserts that the application of state disciplinary proceedings has been “harsh” by holding attorneys responsible who do not fully grasp the potential pitfalls of AI, especially solo practitioners and those at small firms.
“The state bar may not fully understand AI yet,” says Cha, who is based in Orange, California.
But David Majchrzak, immediate past president of the Association of Professional Responsibility Lawyers, suggests there is nothing remarkable about the proposed changes to California’s rules.
“The rules will remain exactly the same. The comments are just being updated to remind lawyers how they are applied in this [AI] context and clarify existing obligations,” says Majchrzak, general counsel and partner with Rosing Pott & Strohbehn in San Diego.
George Cardona, the State Bar of California’s chief trial counsel, told the ABA Journal in an email that the committee acted in response to a directive by the California Supreme Court “to consider incorporating any additional guidance that may be warranted in light of recent or upcoming generative AI developments, including lawyers’ use of ‘agentic AI’ tools.”
If you ask lawyers for their best advice on using artificial intelligence in their practices, some will suggest it be avoided at all costs. But despite the many news stories about attorneys getting caught citing hallucinated cases in court filings, there are ways to use the technology cautiously and wisely, lawyers and technologists tell the ABA Journal.
“If you give AI a prompt, and it gives you some stuff that sounds great but is suspiciously specific, ask it some follow-up questions. It could even be something general, like ‘How confident are you in that?’ Ask it to vet its own answers. Where are the holes? What are the problems? Flag the issues and put AI to work.”
—Bobby Williams Jr.,
director at iDiscovery Solutions,
who trains attorneys on proper AI use
“Just like when you’re using Westlaw, you don’t just take the case cite because it came up in your query. You go back and read the cases. You can’t just say, ‘Oh, AI gave me this’ without verifying that the case actually exists.”
—Julie Bennett, litigator/investigator for the Indiana Supreme Court Disciplinary Commission and immediate past president of the National Organization of Bar Counsel
“There has to be a method and strategy in place so that how the authorities are cited—the facts and quotations, the analysis, all of it—was independently verified.”
—Jean Cha, Cha Law Ethics
“The best way to use AI is in the middle of a project to see if I missed something along the way. But it should never supplant our judgment. It’s not infallible and may lack context.”
—David Majchrzak, partner and general counsel
at Rosing Pott & Strohbehn and immediate past president
of the Association of Professional Responsibility Lawyers
“Although the instances of lawyers citing hallucinated cases are the ones that tend to make the news, more nefarious and less reported are the instances where AI mischaracterizes a real case. Ultimately, nothing substitutes for actually reading the cases you cite.”
—David S. Kemp, AI innovation team, Ropes & Gray
“Read every case your colleague on the other side has cited, and if she or he has cited an hallucinated case or an inaccurate quotation, call and ask them to correct the submission. If you don’t—or if you do and the citation isn’t corrected—then you are as responsible as that other lawyer for the inaccuracies.”
—Dan D. Kohane, member, Hurwitz Fine
“Push back on the AI. Challenge it about the information it provides. It may well admit it was wrong, though not always. Don’t be afraid to change the prompt or use the same prompt more than once to see if the results change. Since AI is based on predictions, the same prompt may well surface additional information or errors. You can also ask the AI if it is certain about the data it provided you.”
—Jennifer Ellis, lawyer and legal technology consultant
“Don’t use general-purpose chatbots for caselaw research. Tools like ChatGPT and Anthropic’s Claude generate text by probability, so they produce citations that look real without knowing whether the case exists. They mimic the shape of a citation, not its substance. For research, use tools grounded in actual legal databases. Now, the frontier models like ChatGPT are getting better at avoiding fake cases. But regardless, they should never be relied upon.”
—Josh Kubicki, visiting lecturer,
University of Indiana Maurer School of Law
