AI in law: courts are punishing fake AI citations
AI hallucination cases in court: 120 in May, 712 by December

in May 2025, a researcher's database had logged 120 court cases involving AI hallucinations.
by the end of the year it was around 712. (yes, nearly six times as many in seven months.)
and 2026 has opened with a US federal appeals court fining a lawyer, and one of America's best-known firms admitting a brief was prepared with AI.
AI in law has a verification problem. and it's teaching every law student the same lesson: the tool drafts, YOU sign.
What's happening: courts and AI hallucinated citations
quick explainer first. a "hallucination" is when an AI confidently makes something up. in legal work that usually means a case that doesn't exist, a real case quoted saying something it never said, or an invented quote.
the scale:
→ Damien Charlotin, a Paris-based researcher, tracks court decisions dealing with hallucinated content. Business Insider counted 120 cases in his database in May 2025 → by late December, Bloomberg Law put it at an estimated 712 decisions worldwide, about 90% of them from 2025
and just the last few weeks:
- 18 February: the US Court of Appeals for the Fifth Circuit sanctioned a lawyer $2,500 in Fletcher v. Experian, after a brief with at least 16 suspiciously inaccurate citations, likely caused by generative AI (TCPAWorld). the sanction reportedly would have been smaller if she'd come clean
- 19 February: a California appeals court told lawyers at Boies Schiller Flexner it was considering monetary sanctions, after they conceded a brief had been prepared with AI tools and contained material citation errors (TCPAWorld)
the UK isn't immune either. in June 2025 the High Court dealt with Ayinde v Haringey, where a pupil barrister (a barrister in training) cited five non-existent authorities, alongside a separate solicitor case.
Dame Victoria Sharp found the contempt threshold met for the barrister but chose not to start proceedings, stressing that this was not a precedent (Legal Cheek). her line for the whole profession: "It is the lawyer's professional responsibility to ensure the accuracy."
What the sanctions actually teach about verification
look at the pattern across these cases:
- the AI isn't on trial, the lawyer is. no court sanctioned a chatbot. they sanctioned the person who signed
- the cover-up is worse than the error. in Fletcher, not owning up reportedly made the outcome worse
- big firms aren't immune. Boies Schiller has every resource going, and its controls still failed
- juniors are exposed. the Ayinde barrister was described as extremely junior, and the judgment called on heads of chambers and managing partners to make sure their people understand their duties
the same thing shows up in other fields. in medicine, chatbots knew the right answer but users still got it wrong. knowing isn't checking.
What law school won't teach you about legal AI
law school trains you to find authority. it rarely trains you to DISPROVE authority someone hands you.
what's missing:
- citation checking at speed, in a proper legal database rather than a search engine
- reading the actual judgment, not a summary of it
- knowing your duty: the High Court was clear you can't even rely on your own client for citation accuracy, let alone a tool
- disclosing AI use wherever your court or firm requires it
The critical-thinking move: a 5-step verification routine
for any AI-assisted legal research, every single time:
- does it exist? pull every case from a recognised law report or database. no record, no citation
- does it say that? read the paragraph being relied on. a real case misquoted is still a fake authority
- is it still good law? check it hasn't been overturned or doubted since
- is the quote word for word? match every quotation against the original
- could I defend it out loud? if a judge asked where it came from, could you answer without opening a chatbot?
it's the same discipline we argued for with junior developers using AI: treat output as a draft you're accountable for.
What to do this month
- run a citation hunt. ask a chatbot for five cases on a topic you're studying, then try to verify each one in your university's legal database. count the fakes
- read one of these decisions in full. the Ayinde judgment is free on the National Archives' Find Case Law service
- ask about AI checks at vac schemes and placements: how do juniors verify AI-assisted work, and who signs it off?
- build a verification checklist you use on every essay. it'll be muscle memory by your training contract
our guide to how students use AI at uni covers staying on the right side of module rules, and there's more legal coverage on our law hub.
so TLDR: courts don't care who drafted it, only who signed it, so verify every citation like your career depends on it (because it does).
have you ever caught an AI inventing a case, and how did you spot it?
Sources
- AI hallucinations in court documents are a growing problem, and data shows lawyers are responsible for many of the errorsBusiness Insider (via Yahoo News) · 27 MAY 2025
- AI-Faked Cases Become Core Issue Irritating Overworked JudgesBloomberg Law · 29 DEC 2025
- The latest news on law firms being sanctioned for using GenAI: Boies Schiller partner faces sanctions, along with an attorney that really pushed her luck with the Fifth CircuitTCPAWorld · 23 FEB 2026
- High Court warns lawyers over AI use after 'fake' cases cited in submissionsLegal Cheek · 06 JUN 2025
- Ayinde v London Borough of Haringey; Al-Haroun v Qatar National Bank [2025] EWHC 1383 (Admin)The National Archives, Find Case Law · 06 JUN 2025
Get the next issue by email
theVANTAGEd is written by students to bring critical thinking to AI across real industries. One issue every two weeks.
