More than 600 court filings have cited AI-invented case law. Judges are sanctioning attorneys. The flaw isn't user error. It's the same architectural defect that breaks AI analytics tools.
In October 2025, a California attorney named Amir Mostafavi was fined $10,000 after a state appeals court discovered that 21 of the 23 case quotations in his opening brief had been invented by ChatGPT. The same month, a Cronkite News tracker counted more than 600 cases nationwide where lawyers had submitted court filings citing nonexistent legal authority: fake judges, fake rulings, fake page numbers in real law reporters. By December, the Chicago Housing Authority's defense firm Goldberg Segalla was sanctioned nearly $60,000 for the same offense. None of these lawyers were trying to deceive the court. They asked a chatbot for research, and the chatbot delivered confident, formatted, professional-looking citations that pointed to nothing.
The pattern is so consistent it now has a name: lawyers call them "ghost citations." A senior partner asks ChatGPT or Claude for case law supporting an argument. The model returns a list of cases, each with a plaintiff name, a citation number, a court, and a year. Several of the cases are real. Several are completely fabricated. They look identical. The lawyer, working under deadline, files the brief. Opposing counsel, or worse, the judge's clerk, pulls up the citations and finds blank pages.
In May 2025, two California law firms were ordered to pay $31,100 for "bogus AI-generated research." A Wyoming case sanctioned Morgan & Morgan attorneys for eight nonexistent citations. One ruling went further still: it sanctioned a lawyer not for filing fake citations himself, but for failing to detect the fake citations in his opponent's filing. The standard, in other words, is no longer just "don't trust AI blindly." It's "actively assume AI output contains fabrications and verify everything."
Judges have started building this expectation into their courtroom procedures. Several US federal judges now require attorneys to attest that any AI-assisted research has been independently verified against primary sources. The legal profession has, in effect, invented a new compliance layer to compensate for a known architectural defect in the AI tools their members use.
A large language model doesn't store facts. It stores statistical patterns about how language flows. When you ask for a case citation, the model is doing the same thing it does when asked for the next word in a sentence: producing the most plausible-looking continuation given everything it has seen before. "Smith v. Jones, 412 F.3d 555 (9th Cir. 2003)" is a plausible-looking continuation. So is "Smith v. Jones, 487 F.3d 219 (5th Cir. 2007)." So is any number of permutations that don't exist.
The model has no internal flag for "this is a real case I have read" versus "this is a string of tokens shaped like a real case." Both feel equally fluent to produce. And because the model is trained to be confident and helpful, it does not say "I'm not sure if this case is real." It presents both with the same authoritative tone.
This is exactly the same failure mode that haunts AI analytics tools that let the model state a number without showing where it came from. The model runs some queries, narrates a finding, and confidently states a figure. Was that figure actually in the rows the query returned? Or did the model interpolate? With no way to open the underlying query and check, there is no way to tell. The failure shows up the same way it does in legal filings: decimal slips, invented names, and figures that aren't in the underlying data, all delivered in the same confident tone as the correct ones.
Askalytics flags any number in an AI answer that can't be traced back to a real row in your warehouse.
The legal profession's response, "verify every citation against a primary source before filing," is correct, but it's a human process bolted onto a broken tool. The better fix is to build the verification into the system itself, so that every number the AI reports can be traced to the exact query that produced it, and a second AI re-checks the conclusion before it reaches you.
Askalytics' AI Insights pipeline is designed around exactly this principle, applied to analytics rather than law. When you ask for an insight, the AI queries your own data directly and runs the queries it needs to answer the question. The difference is that every number it returns traces back to the exact query that produced it. You can open the receipt for any figure, the Advanced Breakdown, and see the query, the rows, and the source table that the number came from. On top of that, every AI Insight is re-checked by a second, different AI model, so a confident claim that the data does not actually support gets flagged before it reaches you.
The legal profession, in other words, is currently doing manually what well-architected AI tools do automatically. Every Smith-v-Jones-style number linked back to the exact query that produced it, and a second opinion before it ships. The lawyers getting sanctioned are using AI tools that simply don't have that step built in.
Lawyers are early indicators here, not anomalies. They handle high-stakes documents under time pressure, and they're being publicly sanctioned in numbers high enough to make the news. The same architectural flaw, language models that confidently produce unverifiable claims, exists in every AI product that touches business decisions. The difference is whether the tool is honest about its limits or hides them behind a confident voice.
For an analytics-specific take, see how the Confidence Check works: the second-model verification layer that double-checks our own narrator's output before it reaches you.