California lawmakers have approved what is being described as a first-of-its-kind state law aimed squarely at attorneys who lean on generative AI tools to research and draft court filings — and then fail to check the machine's work. The measure, Senate Bill 574, cleared the state legislature on August 31 and now heads to Governor Gavin Newsom's desk, where he can sign or veto it.
The bill, introduced by state Senator Tom Umberg, who chairs the California Senate's judiciary committee, would require lawyers to disclose when they use AI to prepare documents submitted in court, take reasonable steps to verify the accuracy of anything AI-generated — including case citations — and correct false or hallucinated material before it reaches a judge, according to Reuters. The measure also bars attorneys from entering confidential or other nonpublic client information into certain generative AI systems, and it explicitly prohibits delegating the practice of law to AI altogether. Under the technical text tracked by LegiScan, the bill adds Section 6068.1 to the state's Business and Professions Code and amends Code of Civil Procedure Section 128.7 so that submitting attorneys must personally read and verify every cited legal source.
Umberg told Reuters that lawyers continue to submit materials containing hallucinations or other anomalies and blame them on AI, adding that the integrity of litigants and advocates is essential to the courts and judiciary. He has framed the bill as building on existing state civil-litigation rules, which already require filings to be backed by real law, and modeled it after a California Judicial Council rule that governs AI use by judges and court staff.
A Pattern of Fabricated Citations Pushed the Issue
The legislative push didn't emerge from nowhere. In September 2025, California's Second District Court of Appeal issued the state's first published opinion addressing AI hallucinations in Noland v. Land of the Free, L.P., sanctioning an appellant's attorney $10,000 for submitting briefs containing 21 fabricated or erroneous legal citations, according to McGuireWoods. The court said it published the ruling specifically as a warning that attorneys have a non-delegable duty to verify every source they cite.
That case was far from isolated. Tracking by legal research organizations found that more than 600 court cases nationwide had involved attorneys filing AI-hallucinated citations by early 2026, a pace that escalated from a handful of incidents per month to multiple cases a day, per ILS. Hoodline has separately tracked several of these episodes locally, including State Farm lawyers admitting AI invented fake cases in a Carson fire lawsuit, an SF attorney facing scrutiny over phantom AI-generated quotes in a Trump layoffs court fight, and a San Diego custody appeal where a court sniffed out fake citations while still imposing $5,000 in sanctions and a State Bar referral.
Where Software Ends and 'Practice of Law' Begins
Not everyone agrees the new statute breaks meaningful new ground. Wayne Stacy, executive director of the Berkeley Center for Law and Technology at the University of California, Berkeley School of Law, said the measure is largely duplicative of existing ethics rules already covering lawyers, per Reuters. Judges have already sanctioned litigants for including erroneous or fabricated AI-generated material in filings without verification, and SB 574 mainly hands courts another tool for penalizing lawyers who don't follow the law.
Still, the bill's non-delegation clause has stirred debate among legal technology analysts. Proposed Section 6068.1(a)(2) states plainly that an attorney shall not delegate the practice of law to generative AI, but Artificial Lawyer notes that California courts have spent more than a century defining what counts as the practice of law in the first place, leaving open questions about where routine AI-assisted research ends and prohibited delegation begins.
Rules Extend to Arbitrators, Not Just Courtroom Lawyers
SB 574 doesn't stop at litigators. The bill also prohibits neutral arbitrators from delegating decision-making to generative AI models and bars them from relying on AI-generated information outside the official case record without disclosing it to all parties, according to Farella Braun + Martel LLP. Lawmakers were reportedly concerned that arbitrators might feed private evidence into commercial AI platforms to help draft binding decisions.
The legislation also sits alongside other regulatory layers already in place. Effective September 2025, California Rule of Court 10.430 required state courts to adopt written generative AI policies for judges and court staff, or ban the technology outright, by December 2025, per LegalTechMag. Judges and research attorneys remain strictly barred from using AI to analyze evidence or draft opinions under those judicial rules. The California Judicial Council, the policy-making body for the state's court system, has taken no position or public comment on SB 574 itself.
Separately, the State Bar of California has been moving on a parallel track. In March 2026, the bar's Standing Committee on Professional Responsibility and Conduct approved amendments writing explicit AI obligations into six existing Rules of Professional Conduct, following an August 2025 directive from the California Supreme Court, according to the Daily Journal. The bar also updated its practical guidance in May 2026 to address autonomous “agentic AI” tools capable of executing multi-step legal tasks with minimal human oversight.
Sanctions Can Trigger State Bar Investigations
The stakes for lawyers who get caught go beyond a single judge's ruling. Under California Business and Professions Code Section 6068(o)(3), courts and attorneys must report judicial sanctions exceeding $1,000 to the State Bar of California, which automatically triggers disciplinary investigations, according to California Legal Ethics. Appellate courts that have issued $5,000 to $10,000 sanctions for AI errors have routinely sent referral orders directly to the bar's Office of Chief Trial Counsel.
California's approach also stands out nationally. While the state chose to legislate the issue outright, other jurisdictions such as New York implemented centralized court system rules governing attorney AI use back in June 2026, and no single Federal Rule of Civil Procedure on AI exists nationwide, per LegalTechMag. Federal district courts still rely on a patchwork of individual judge standing orders, though a formal Federal Rule 11 amendment was proposed in May 2026.
For now, the bill's fate rests with Newsom. His office did not respond to a request for comment from Reuters as of September 1. If signed, supporters argue the law's verification requirement should raise the standard for lawyers submitting material in court — a bar that, if California's recent run of hallucination cases is any indication, more than a few attorneys have already failed to clear.
.jpg){kind=link}