Can generative AI prompts be used for evidence?
Photo illustration by Sara Wadford/Shutterstock
“Hey, ChatGPT, how can I launder money?” “What are the ingredients in crystal meth?” “Best ways to destroy evidence.”
As Open AI’s ChatGPT ushers in an explosion of interest in generative AI, e-discovery experts are considering how user-entered prompts like the ones above can be obtained and if they would hold up in court.
Open questions include how to access generative AI prompts; what information about timing and users can be acquired; how strong protections in contracts with bespoke commercial generative AI providers are; and even what qualifies for attorney-client privilege.
“Nobody knows, and there’s no good answer,” says Maura Grossman, a research professor in the computer science school at Canada’s University of Waterloo.
Ironically enough, OpenAI already has been hit with a lawsuit in which ChatGPT prompts and the resulting responses have been offered as evidence by plaintiffs.
In federal complaints filed against Meta and OpenAI, comedian Sarah Silverman and authors Christopher Golden and Richard Kadrey accuse the companies of copyright infringement by ingesting their work to train their AI programs.
Exhibits in the suits, for in stance, include the plaintiffs’ own prompts, showing that ChatGPT can summarize their books—including Silverman’s 2010 memoir The Bedwetter—proving they are part of the OpenAI database. According to the OpenAI complaint, “At no point did ChatGPT reproduce any of the copyright management information plaintiffs included with their pub- lished works” or receive the au- thors’ consent.
The Meta complaint states that the company “intentionally removed CMI from the plaintiffs’ infringed works.”
The authors are seeking statutory damages, an injunction, restitution of profits and additional damages.
They’ve met plenty of pushback. In November, a federal judge granted Meta’s request to dismiss all claims except the one alleging that unauthorized copying of the plaintiffs’ books to train the large language model constitutes copyright infringement.
OpenAI has asked the court to dismiss parts of the lawsuit, saying the ChatGPT text doesn’t violate the authors’ rights. The company does not comment on active litigation, a company spokesman said. Meta did not respond to a request for comment.
Search and seizure?
Knowing whether a defendant in a criminal case entered the prompt “best way to strangle someone” into the OpenAI platform could be helpful in building a case, Grossman says.
But how would those prompts be obtained?
The Stored Communications Act protects the privacy of internet service providers’ files and their subscribers’ information, such as billing records or IP addresses. It applies to social media platforms.
But it’s unclear if it will apply to the consumer-grade free ChatGPT in a criminal case, says Grossman, a member of the ABA Law Practice Division.
“A reasonable argument could be made that they’re no different than Facebook or any of these other social media sites,” she says, “in which case, you would not be able to get that information by subpoena. You’d either have to get consent of the person who has the account with OpenAI, or you’d have to get a warrant.”
Civil cases could face a similar roadblock under the act, she says, requiring the account holder’s consent to obtain the messages’ content.
Problematic preservation
Even if ChatGPT conversations can be obtained, Craig Ball, an Austin, Texas-based attorney, professor and electronic evidence expert, questions whether they’d pass the court’s sniff test.
The consumer-grade generative AI chatbot automatically saves conversations between the program and the user on the left side of the window. That data is used to improve OpenAI’s language model.
When Ball exported his own history from ChatGPT during a video call with the ABA Journal, he found all chats were included. However, the export did not include proper timestamps or the name of the user.
“That is problematic from a preservation standpoint, since what I knew and when I knew it is going to be typically more important from the standpoint of evidence,” he says.
It also makes validation tricky. “How do we trust its usage? If somebody produces this in e-discovery, is it something the user may have themselves collected and generated?” Ball asks. “I don’t have the answer to that.”
ChatGPT settings include workarounds to turn off the chat history and data training. Once a toggle is turned off, unsaved chats will be deleted within 30 days. Also, chats can be deleted and accounts closed.
For those with enterprise generative AI commercial licenses, contracts might include how long information is preserved. The system at Merlin Search Technologies, an AI and cloud technologies e-discovery software company, runs off a commercial license, Merlin CEO John Tredennick says. The company’s agreements specifically state it will not save prompts or use them to train the system, “and they don’t,” he says. “So it’s not a risk—if you’re working through a commercial license—for lawyers and law firms to use the tool.”
Still, Grossman is not sure what will happen if that agreement is tested by a lawsuit.
“Most of those contracts say that whoever’s providing you with the AI won’t access it,” Grossman says. “We’ll put that in brackets because who knows? When the government comes asking, things change.”
But Ball says the more protections in place, the better. “Obviously the less evidence that remains, the less easy it is to hold someone accountable for their misbehavior,” he adds.
Attorney-client privilege
As more people consult generative AI for legal matters, issues of attorney-client confidentiality could arise, Ball says.
“What if your OpenAI becomes your 21st century lawyer? Are those interactions entitled to any form of traditional privilege?” he asks. “How intelligent does artificial intelligence need to be till we extend to it some of the same ambit of confidentiality that we would give to human interactions?”
Grossman disagrees. “A privilege exists between a client and a lawyer, and AI is not a lawyer. AI did not take the bar exam, pass the bar exam and get licensed to practice law,” she says. “Privilege assumes a person, a human attorney.”
The debate around generative AI as evidence will continue, experts agree.
“These are all things that need to be tested and verified,” says Mary Mack, CEO and chief legal technologist of Electronic Discovery Reference Model. “These are heady days.”
This story was originally published in the February-March 2024 issue of the ABA Journal under the headline: “Raising Questions: Can generative AI prompts be used for evidence?”
