'Don't blame your associate' for your AI mistakes or hallucinations
Cases involving lawyers who introduced artificial intelligence-generated mistakes are popping up all over the country.
In Judge Roy B. Ferguson’s experience in the 394th Judicial District Court of Texas, he’s had lawyers read what they say is a state statute from their phones, but “it’s wrong, just flat wrong.”
“I said, ‘Can I see what you’re reading from?’” said Ferguson, noting that it turned out to be ChatGPT or another AI tool. “‘Yeah, that’s what I thought. Stop going to AI and arguing it in this courtroom.’”
Ferguson and Robert Hawkins, an attorney with Paykin Law in New York, discussed this issue Friday during an ABA Techshow program titled “Hallucination Nation: Catching Fake AI Cases Before They Tank Your Brief.”
Hallucinations appear most in content generated by ChatGPT and other free generative AI tools because they draw from large data sources, including internet sites like Reddit where people can post false information, Hawkins said. These tools also entice users to continue asking questions, and over time they learn the kinds of responses users want to receive, he said.
“It’s the sycophant problem,” Hawkins said. “These large language models want to please you. They want to make you happy. They want to give you what you want.”
To avoid this problem, Hawkins suggested lawyers use paid versions of tools that retrieve their information from limited datasets. For instance, a lawyer could upload the Texas state statutes into their tool’s library and prompt it to pull from those when conducting research.
Follow along with the ABA Journal’s coverage of the ABA Techshow 2026 here.
Ferguson also pointed out that AI tools don’t understand jurisdictional weakness or other complex legal concepts. They might provide caselaw that is applicable in San Antonio but not El Paso, he said. They also could cite federal law instead of state law when generating responses.
“It is true that these platforms are great at pulling information from datasets,” said Ferguson, a 2025 ABA Journal Legal Rebel. “But if you expect them to be as smart as the best lawyers in the country at evaluating complex webs of law and coming to a correct conclusion, they’re not. That’s where you have to come in.”
A lawyer’s duty of candor and competence requires them to verify every piece of information they submit to the court, Hawkins said. And if they do submit a brief with a hallucinated case, he said they should immediately write to the court to acknowledge their mistake.
“Falling on your sword is the best option,” Hawkins said. “Don’t wait until the oral argument. Don’t wait until you’re standing in front of the judge.” He added that lawyers should “be prepared to accept the consequences for the mistake,” and if they do, the penalty or sanction should be “be a lot lighter.”
Ferguson agreed, adding that most judges understand that mistakes happen and are more likely to show leniency to lawyers who own those mistakes.
“You can’t pretend that it didn’t happen, and you can’t blame anyone else,” Ferguson said. “Don’t blame your staff. Don’t blame your associate. Don’t blame the draft. Don’t blame the clerk.”
See also:
Lawyer’s Zoom hearing is a purr-fect storm tech glitch
Write a letter to the editor, share a story tip or update, or report an error.
