The Chat Box Is Not a Vault:
August 2026 · VdVLaw
When Clients and Staff Start Using AI on Their Own
Somewhere right now, a well-meaning client is pasting a police report into an AI chatbot.
The chatbot is being extremely helpful. It is summarizing, organizing, identifying possible defenses, and transforming twelve pages of human panic into a document with headings, bullet points, and the quiet confidence of someone who has never been cross-examined.
A few minutes later, the client emails the lawyer a polished memorandum: “I thought this might save you some time.”
It might. It might also omit the sentence on page 47 that changes everything, merge two witnesses into one remarkably busy person, or convert a disputed allegation into an established fact because the police report used a firm tone. The chatbot does not know it did any of this. The chatbot does not know anything. Knowing things is not its job. Sounding like it knows things is its job, and business is booming.
Still, the document looks excellent. It has bold type.
Clients are not the only ones doing this. Inside law offices, staff members are using AI to summarize discovery, build timelines, clean up correspondence, and turn a stack of medical records into something that can be read before retirement.
Nobody is being careless on purpose. That is exactly what makes it dangerous.
A Federal Defendant Asked Claude for Help
In United States v. Heppner, a man under federal investigation used the consumer version of Anthropic’s Claude after receiving a grand jury subpoena and retaining lawyers. He gave the chatbot information relating to his defense and generated 31 documents, including an outline of a defense strategy based on the charges his lawyers anticipated. Some were shared with his legal team, presumably with a sense of accomplishment.
Think about that workflow for a second. It is the digital equivalent of photocopying your lawyer’s notes and mailing them to a helpful stranger who once read a law blog. Except this stranger has perfect recall, a terms-of-service agreement, and no idea that it is a stranger.
When federal prosecutors obtained the materials, he claimed attorney-client privilege and work-product protection. In February 2026, Judge Jed Rakoff of the Southern District of New York rejected both arguments. The work-product claim failed because the documents had not been prepared by counsel or at counsel’s direction, and the court focused on the lack of confidentiality surrounding the consumer product used. As for whether a chatbot can stand in for counsel, the court needed only four words to describe Claude:
“Plainly not an attorney.”
Judge Jed Rakoff, United States v. Heppner (S.D.N.Y. 2026)
Four words. Lawyers bill in six-minute increments, and that still would have been the cheapest legal advice in the entire case.
The result was not that every AI conversation everywhere is automatically discoverable. The result was narrower and more unsettling: a client had created organized summaries of his own defense, using information from his own lawyers, and those summaries were available to the government.
The terms of service, a document read in full by approximately no one, turned out to matter quite a lot.
Somewhere out there, the attorney who drafted paragraph 14.7 of a consumer privacy policy felt a sudden warm glow and could not explain why.
Then Another Court Went the Other Way
That same week, a federal court in Michigan reached a different result in Warner v. Gilbarco. A self-represented civil litigant had used ChatGPT to help draft filings and analyze case materials. The defendants demanded her AI prompts and responses. The court refused to compel them, concluding that the materials reflected her mental impressions and were protected as work product. Its description of generative AI programs was three words long:
“Tools, not persons.”
Warner v. Gilbarco, Inc. (E.D. Mich. 2026)
That distinction mattered because work-product protection is generally waived only when material is disclosed to an adversary, or in a way likely to place it in an adversary’s hands. On the record before that court, ChatGPT was not the plaintiff’s adversary.
So within one week, one federal court treated an AI chat like a conversation shouted across a crowded bus terminal, and another treated it like a legal pad. Both courts had reasons. Neither had company.
Heppner and Warner are not identical cases, and their reasoning does not create a tidy national rule. That is precisely the point. A client who uploads case information may not know which doctrine will apply, which court will decide it, or whether the account settings and product terms will change the result.
Privilege should not depend on a coin flip conducted after the client has already pressed Enter. A coin flip is fine for deciding who kicks off. It is less fine when heads means work product and tails means the government reads your defense strategy over coffee.
Even the Man Who Sells It Says So
If this sounds like lawyerly hand-wringing, consider who delivered the clearest warning on record. In the summer of 2025, OpenAI’s own chief executive said this out loud, on a podcast:
“So if you go talk to ChatGPT about your most sensitive stuff and then there’s like a lawsuit or whatever, we could be required to produce that, and I think that’s very screwed up.”
Sam Altman, CEO of OpenAI, This Past Weekend with Theo Von (July 2025)
He noted that conversations with a therapist, a doctor, or a lawyer carry legal privilege, and that “we haven’t figured that out yet for when you talk to ChatGPT.”
When the person whose company makes the product tells you the product is not confidential, that is not a disclaimer. That is the manufacturer reading you the warning label out loud. It is the head of a parachute company mentioning, mid-flight, that the industry is still working on ripcords. Interesting information. Better timing was available.
Delete Does Not Mean Gone
The settings matter, too. OpenAI says personal users can turn off model training, that Temporary Chats are deleted within 30 days, and that business and enterprise accounts are not used for training by default. Those protections are real. They are also settings, not guarantees, which is why “the office uses ChatGPT” tells you almost nothing. Which version? Whose account? Through an approved business workspace, or through the free account someone created three years ago to generate a birthday poem for an uncle nobody actually likes?
And a court order can override all of it. In the New York Times copyright litigation against OpenAI, a federal magistrate judge ordered OpenAI in May 2025 to preserve output logs it would otherwise have deleted, including chats users thought they had erased. In November 2025, the court ordered production of a sample of 20 million de-identified user conversations, and in January 2026 the district judge affirmed that order.
Sit with that number. Twenty million conversations from ordinary users, headed into a lawsuit between two companies those users have never met, will never meet, and did not know were fighting. Within that litigation, the delete button turned out to be less a shredder and more a decorative element. It is the elevator close-door button of the internet.
The Client’s Summary May Be Useful
None of this means a client who uses AI has ruined the case. Panic is rarely an effective preservation strategy.
AI can help a client organize dates, identify questions, translate unfamiliar language, or understand the basic structure of a legal process. A frightened person facing criminal charges at two in the morning is not going to say, “I shall wait patiently until counsel’s office opens.” The client is going to search. Increasingly, the search box talks back. It talks back warmly, instantly, and at 2 a.m., which is three more things than most law offices offer.
The problem is not that the resulting summary is always wrong. The problem is that the lawyer may have no reliable account of how it was made. What documents were uploaded? Were all pages included? What instructions were given? Did the chatbot fill gaps? Did the client edit the response? Was the answer shared with anyone else?
A clean summary can conceal a dirty process.
That matters because legal files often turn on the part that refuses to summarize neatly: a three-second pause, a shifted timestamp, a pronoun with no clear antecedent, a laboratory value in an unfamiliar unit, or a witness whose certainty improves dramatically between the first interview and the third.
Generative AI is rewarded for producing coherent answers. Litigation is frequently built from the discovery that the answer was never coherent in the first place. The messier the file, the smoother the summary, which is exactly backward, like a smoke detector that gets quieter as the fire spreads.
The Cat Has Brought You a Memorandum
When a client delivers an AI-generated case analysis, the lawyer should treat it like a cat presenting a dead bird.
The cat is proud. The cat expects recognition. The cat does not understand why the presentation has raised several procedural questions. The cat has never heard of spoliation, and if it had, the cat would not care.
Thank the client for trying to help. Then ask what was uploaded, which service and account were used, whether the conversation still exists, what instructions were given, whether the client edited the output, and whether anything was shared or deleted.
Do not begin by announcing that the client has destroyed privilege, contaminated the evidence, and placed the entire defense on a server farm in an undisclosed location. Apart from being unhelpful, it may not be true.
Preserve the facts first. Determine the consequences second. The lecture can wait. The lecture always waits. The lecture has never once improved by being delivered early.
Meanwhile, Down the Hall
Clients are only half of the problem. The same thing is happening inside law offices, usually with the best intentions.
A paralegal builds a chronology by pasting discovery into a free AI tool. An assistant summarizes medical records because the stack is four inches tall and the afternoon is short. Someone improves a difficult client email in a personal account because the office-approved software requires nine clicks, two passwords, and the recovery code from a phone that belonged to an employee named Linda. Linda left in 2019. The recovery code left with her.
Calling this laziness misses the point. People use tools that make unpleasant work easier. And on the confidentiality question, the State Bar of California has already said the quiet part in writing:
“A lawyer must not input any confidential information of the client into any generative AI solution that lacks adequate confidentiality and security protections.”
State Bar of California, Practical Guidance for the Use of Generative Artificial Intelligence in the Practice of Law
The updated 2026 version of that guidance applies the familiar duties to generative and agentic AI alike: competence, confidentiality, supervision, communication, candor, and protection of client information. ABA Formal Opinion 512 takes the same basic approach under the Model Rules. Lawyers remain responsible for understanding the tool, protecting information, supervising staff and vendors, checking the work, and communicating with clients when appropriate.
“I didn’t know the paralegal was using ChatGPT” is not a defense. Under the rules of professional conduct, it is closer to a confession.
Adding artificial intelligence does not remove the lawyer from the workflow. Disappointing, perhaps, but consistent with the general structure of professional responsibility, which has been ruining shortcuts since before electricity.
“Do Not Use AI” Is Not an AI Policy
A blanket prohibition may feel safe. It is also likely to produce secret use.
If a tool can turn an hour of tedious work into ten minutes, someone will eventually try it. If the office offers no approved alternative and no practical instructions, the employee may use a personal account and simply avoid mentioning it. Prohibition does not eliminate behavior. It eliminates conversation about the behavior. Ask anyone who has ever posted a sign that says absolutely no food in the break room.
Now the office has two problems: unapproved AI use and an employee who believes honesty about it is professionally unwise.
“Use common sense” is not much better. Common sense differs significantly between the attorney who sees confidential work product and the employee who sees a grammar tool with a friendly blinking cursor.
A usable office policy does not need to explain the history of machine learning or include a diagram that looks like the wiring plan for a submarine. It should identify approved systems and accounts, categories of information that may not be entered, when attorney approval is required, how outputs must be checked, where AI-assisted work should be saved, and how mistakes should be reported without the employee first retaining separate counsel.
The policy should be clear enough to guide someone at 4:45 on a Friday afternoon. That is when policies discover whether they are real.
The First Question Is Not “Did You Use AI?”
That question sounds accusatory and produces an immediate desire to become less specific. The better question:
What information did you give it?
Maybe the client entered only public facts and asked for a definition. Maybe a staff member used an approved business system to reorganize a fully redacted document. Or maybe someone uploaded the police reports, medical records, defense strategy, witness addresses, and a photograph of the office Wi-Fi password because the chatbot also offered to troubleshoot the printer. It is always willing to troubleshoot the printer. It has never fixed a printer. Nothing has ever fixed a printer.
Those are different events. The word “AI” alone does not describe the risk. The information, platform, account, settings, purpose, and human review do.
The Ineffective-Assistance Wave
Here is my prediction, and I do not think it is a bold one. Ineffective-assistance-of-counsel claims built on AI are coming, and the first wave has already hit the beach. Pras Michel of the Fugees sought a new trial arguing, among other things, that his lawyer’s use of an AI tool to help draft the closing argument amounted to ineffective assistance, after that closing misattributed a Puff Daddy lyric to the Fugees. In front of a jury. In a federal criminal trial. While representing a member of the group that actually recorded the song.
The judge denied the motion. But the theory has now been filed, briefed, and ruled on. Every unverified AI filing, undisclosed client upload, and unsupervised staff workflow that touches a criminal case is a future claim waiting for an appellate lawyer with excellent search terms and no weekend plans.
Not because AI is bad. Because unexamined AI use is exactly the kind of mess that walks into court beautifully dressed, perfectly composed, and completely unable to explain where its citations came from.
AI Is Not the Villain
Generative AI can be very useful in legal work. It can assist with organization, preliminary timelines, drafting, translation, issue lists, document classification, and the many administrative tasks that consume time without necessarily requiring a law degree.
The answer is not to build a moat around the office and hope artificial intelligence respects property boundaries. It does not respect property boundaries. It does not know what property is. It has read every deed ever written and retained none of the point.
It is already inside. It is on the client’s phone, the receptionist’s browser, the investigator’s laptop, and the tab someone minimizes whenever the managing attorney walks past.
The real distinction is no longer between offices that use AI and offices that do not. It is between offices that know how AI is being used and offices that prefer not to ask.
A client only has to be frightened. A staff member only has to be overworked. A chatbot only has to be useful. And everyone only has to assume that somebody else already checked.
The chat box is not a vault. It may be an excellent tool. But before anyone places the case inside it, someone needs to know whose tool it is, what it does with the contents, and whether the door actually locks. The people who figure that out now will get to read about the people who figure it out later. In published opinions. With footnotes.
I have developed a client handout and intake questionnaire on this issue for attorneys to adapt to their own practices. I am also awaiting State Bar of California approval for an MCLE course addressing the larger practical and ethical questions. Apparently, “please stop putting the entire case into the free chat box” now requires both written materials and continuing education.
VdVLaw provides attorney-directed legal support and consulting services. VdVLaw is not a law firm and does not provide legal advice. Nothing in this article is legal advice.
Sources and Further Reading
United States v. Heppner, No. 25-cr-00503-JSR (S.D.N.Y., bench ruling Feb. 10, 2026, written opinion Feb. 17, 2026). AI-generated documents held not protected by attorney-client privilege or work product. Paul, Weiss: https://www.paulweiss.com/insights/client-memos/sdny-court-considers-whether-ai-generated-documents-are-subject-to-privilege-protections; Harvard Law Review Blog: https://harvardlawreview.org/blog/2026/03/united-states-v-heppner/
Warner v. Gilbarco, Inc., No. 2:24-cv-12333 (E.D. Mich. Feb. 2026). Pro se plaintiff's AI materials protected as work product; AI platforms are “tools, not persons.” Proskauer Rose: https://www.proskauer.com/alert/michigan-federal-court-protects-ai-assisted-litigation-work-product; Perkins Coie: https://perkinscoie.com/insights/update/heppner-and-gilbarco-courts-apply-privilege-and-work-product-protection-generative
Sam Altman remarks on ChatGPT and legal confidentiality, This Past Weekend with Theo Von (July 2025). TechCrunch: https://techcrunch.com/2025/07/25/sam-altman-warns-theres-no-legal-confidentiality-when-using-chatgpt-as-a-therapist
New York Times v. OpenAI preservation and production orders (S.D.N.Y. 2025-2026). ABA Journal: https://www.abajournal.com/news/article/chatgpt-creator-must-turn-over-20m-chat-logs-in-copyright-litigation-federal-judge-says; National Law Review: https://natlawreview.com/article/when-chats-become-evidence-court-affirms-order-requiring-openai-produce-20-million
American Bar Association, Formal Opinion 512, Generative Artificial Intelligence Tools (July 29, 2024): https://www.americanbar.org/content/dam/aba/administrative/professional_responsibility/ethics-opinions/aba-formal-opinion-512.pdf
State Bar of California, Practical Guidance for the Use of Generative Artificial Intelligence in the Practice of Law (Nov. 2023, updated May 2026): https://www.calbar.ca.gov/Portals/0/documents/ethics/Generative-AI-Practical-Guidance.pdf; Ethics and Technology Resources: https://www.calbar.ca.gov/legal-professionals/legal-resource-center/ethics/ethics-technology-resources
OpenAI data handling: Enterprise Privacy, https://openai.com/enterprise-privacy/; Data Controls FAQ, https://help.openai.com/en/articles/7730893-data-controls-faq; Temporary Chat FAQ, https://help.openai.com/en/articles/8914046-temporary-chat-faq
United States v. Michel, motion for new trial denied (D.D.C. 2024). Canadian Lawyer: https://www.canadianlawyermag.com/news/international/us-judge-denies-rapper-pras-michels-request-for-new-trial-despite-ai-error-in-closing-argument/388627
