Attorneys and AI: Watch Out for Trouble
“If it were ever an excuse to plead ignorance of the risks of using generative AI to draft a brief without verifying its output, it is certainly no longer so.”
– Fletcher v. Experian Info Solutions, No. 25-20086 (5th Cir. Feb. 18, 2026)
It’s past time to sound the alarm on the improper use of generative AI in the legal field.
Over the last three years, numerous attorneys, law clerks, pro se litigants, and others have gotten in trouble for filing documents with the court that contain fabricated and unverified material generated (“hallucinated”) by AI.
Just last month, the Supreme Court of Connecticut released an order (dated July 31, 2026; PDF) sanctioning an attorney for submitting filings that cited entirely fabricated cases. This came two weeks after the state’s Rules of Appellate Procedure were amended to require independent verification of all AI-produced citations and legal authorities (PDF).
It’s concerning that such errors could have found their way into one of the highest courts in the country. And that’s just one example – we’ll look at more below. Then we’ll answer the important question: Why does this keep happening?
Courts Sanction Attorneys for Improper AI Use
It’s not the use of generative AI per se that has gotten attorneys in trouble with the courts. It’s the use of AI without verifying the output, which has resulted in numerous filings that contain citations to non-existent cases, made-up quotes, incorrect summaries of existing cases, and the like.
Attorneys caught submitting filings with unverified AI-generated content have faced various sanctions, most commonly additional CLE hours and hefty fines. Many no doubt have suffered reputational damage and career fallout, too. While no attorneys appear to have been disbarred for careless use of generative AI – yet – multiple courts have warned against it.
Here are some notable examples out of hundreds of similar instances:
Mata v. Avianca (2023) – One of the first cases to gain national attention
Attorneys were sanctioned for filing a brief that cited cases, opinions, and quotes entirely fabricated by ChatGPT. Among other sanctions, they were ordered to pay $5,000 and to write letters to all the judges they had falsely cited.
At this time, ChatGPT was barely a year old, and the concept of AI hallucination (see below) not widely known. One of the sanctioned attorneys, Mr. Schwartz, stated, “I just was not thinking that the case could be fabricated… My reaction was, ChatGPT is finding that case somewhere. Maybe it’s unpublished. Maybe it was appealed. Maybe access is difficult to get. I just never thought it could be made up.”
See: Mata v. Avianca Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023); Article from The New York Times
Park v. Kim (2024) – Notable for occurring in a federal appellate court
The attorney here cited a non-existent, AI-generated case in a brief submitted to the U.S. Court of Appeals for the Second Circuit. She was ordered to furnish a copy of the opinion to her client and referred to the Court’s Grievance Panel “Because this conduct falls well below the basic obligations of counsel.” While the sanction was not severe, this case is notable for being an early instance of unverified AI content going before a federal appellate court.
See: Park v. Kim, 91 F.4th 610 (2d Cir. 2024); Article from The National Law Review
Whiting City v. City of Athens, Tennessee (2026) – Attorneys fined $15,000 each
Many attorneys have been ordered to pay a few thousand dollars, but the attorneys in this case faced a punitive sanction of $15,000 each, plus reimbursement of attorney’s fees and other costs, for submitting filings with fabricated and unverified citations and quotes. The U.S. Court of Appeals for the Sixth Circuit had some strong words: “They have brought the profession into disrepute… [They] have sullied the reputation of our bar, which now must litigate under the cloud of their conduct.”
See: Whiting v. City of Athens, No. 25-5424 (6th Cir. 2026) (PDF); Article
Withers v. City of Aberdeen (2026) – Attorneys disqualified from case
Earlier this summer, attorneys on both sides were fined and removed from a case for submitting briefs that included AI-generated mistakes. Two were also barred from appearing before the U.S. District Court for the Northern District of Mississippi for two years. Disqualification has been rare, with judges preferring other sanctions that don’t remove the client’s legal representation.
See: Judge Aycock’s order (PDF); ABA Journal article
A Note on Pro Se Representation
Perhaps unsurprisingly, several similar instances have occurred in legal matters in which a party is representing himself or herself. Pro se litigants should not expect to get off lightly, however, simply because they are not admitted to the bar. Courts can and routinely do sanction these individuals, too.
In one notable example, a man representing himself had his case dismissed and was ordered to pay $10,000 to the Appellant for filing a frivolous appeal largely based on fictitious citations. See: Kruse v. Karlen, 692 S.W.3d 43 (Mo. App. E.D. 2024).
How and Why Does This Keep Happening?
This is happening with concerning frequency primarily because many attorneys and other legal professionals don’t realize how generative AI actually works.
AI chatbots are not just friendly, talkative search engines.
Rather, Generative AI tools generate new content by predicting what is most likely to come next in the sentence.
Popular applications like ChatGPT, Gemini, and Copilot are built on top of large language models (LLMs) which have been trained on enormous amounts of existing text and data. These programs generate new content by recognizing patterns and predicting the next most-likely words. They are not glorified search engines basing output exclusively on existing, verified sources. When context or data are sparse, AI may “hallucinate” and fill in missing gaps with made-up material. (Read more on “Why language models hallucinate” from ChatGPT creator OpenAI.)
This is why, for example, an AI chatbot might include made up citations in a brief. There is no Day vs. Chandler Foods, Inc., 2023 WL 5178921 (E.D. S.C. Jan. 15, 2023), but at first glance it looks like a properly cited case and could easily be left in a filing.
Knowing this can help explain why so many attorneys and other legal professionals have made these kinds of mistakes.
The Bottom Line: Don’t Avoid AI, But Don’t Rely On It
The takeaway for attorneys and others in the legal field: You don’t need to avoid AI entirely but use it as a starting point only. Be sure to start with good input prompts to get good output. Most importantly, make sure to verify all cases, citations, quotations, and other output generated by AI tools.
Gem and his team at the Gem McDowell Law Group handle estate planning, business law, and commercial real estate from their offices in Myrtle Beach and Mt. Pleasant, SC. Call (843) 284-1021.
Artificial Intelligence or Manufactured Intelligence? Why AI Doesn’t Replace an Attorney
Artificial Intelligence (AI) has come a long way in recent years, but as impressive as it is, it has many limitations.
AI cannot and does not replace an attorney, for one.
AI only responds to the input received. One big issue is that the way a question is asked can affect the answer given. If the question isn’t the right question asked in the right way, then the answer cannot be right, it will be biased. A good attorney removes the bias; AI cannot.
It’s easy to see how this could work for medical issues. A person might put in a list of symptoms and the AI chatbot replies with a list of possible problems and diagnoses, not taking into account the person’s medical history or other pertinent information. He or she may believe the root cause is a terrible disease, when in reality it’s an AI-generated error.
Just as AI’s powers can be misapplied in medicine, the same can happen in law.
Real-life examples. A man contacted our office concerned that his business could be found liable for wrongful termination, which could have serious consequences. An ex-employee who was not happy about being fired asked an AI chatbot about his situation, and the AI led the ex-employee to believe his firing was a “retaliatory discharge.”
The ex-employee had the AI chatbot write a letter to the employer. But it became clear that the firing was not a “retaliatory discharge,” which is a particular type of wrongful termination under South Carolina law. The AI had made several assumptions that were not true and had based its arguments on those assumptions. The employer who contacted us was relieved to find out the letter was baseless.
A similar thing happened with another employee of a different business, when a young woman upset over being fired had AI write a four-page letter threatening legal action. The letter looked great, it sounded great, but it was ersatz – four pages of legal arguments and threats built on faulty assumptions and misapplication of the law.
How could this happen? In both cases, the employees didn’t know what they didn’t know. They didn’t know enough to even ask the AI the right question about their situations, so they got wrong answers. We don’t know exactly what prompts were used, but if one had put in something like, “I got fired from my job and I didn’t do anything wrong, is there anything I can do?” an AI chatbot could easily go from there and come up with reasons why the firing was improper and the employee had legal recourse. But it was made up.
The takeaway: Do not rely on AI to give good legal advice. It is not at the point where it can provide legal advice that’s accurate, pertinent, and unbiased.
However, you don’t have to avoid it completely. AI can be a very helpful tool when starting the research process. Just be mindful of the way you phrase questions and input you provide, as that will directly affect the answers you get back. Always verify the AI’s answers with reliable sources, too.
Call Attorney Gem McDowell
For human help with business law or estate planning in South Carolina, call Gem and his team at the Gem McDowell Law Group with offices in Myrtle Beach and Mt. Pleasant, SC, today at (843) 284-1021.
