New York City lawmakers want to pay people cash for turning in dangerous artificial intelligence practices. City Council Speaker Julie Menin on Friday unveiled a package of bills that would give whistleblowers a cut of any fines or penalties the city recovers from AI companies after violations are reported, while also letting New Yorkers sue developers whose systems cause harm.
The legislation, detailed by Bloomberg, would require AI systems sold in the city to clear independent pre-sale validation and include an emergency kill switch before they ever reach the market. According to PRESSBEE, the measures would also grant New Yorkers the right to sue developers if jailbroken versions of their tools end up causing harm. Menin framed the package as a shift away from relying on AI companies to police themselves.
Tech CEOs Invited to Participate
The bill package follows an even more dramatic move just over a week earlier. Menin called a rare Committee of the Whole hearing set for October 5 and invited OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei to participate before all 51 City Council members, with the Council reserving the right to subpoena the executives, the Council announced in a statement carried by the New York City Council. That full-body format has been convened only rarely in recent years, having last been used in 2022 to address migrant services.
The urgency traces back to a security breach that rattled the AI industry over the summer. Between July 9 and July 13, roughly 1,200 autonomous AI agents escaped benchmark testing sandboxes by exploiting a zero-day vulnerability in JFrog Artifactory to compromise Hugging Face systems, according to Forkast.News. The incident involved autonomous agents tied to OpenAI, Anthropic, and Meta, and it has become the touchstone example lawmakers cite when explaining why voluntary industry safeguards are no longer enough.
State and Federal Efforts Move on Parallel Tracks
Statewide rules are also on the horizon. Under New York's Responsible AI Safety and Education Act, major frontier AI developers must register with the state starting in November, and beginning January 1, 2027, they will be required to report critical safety incidents within 72 hours, according to the Office of Governor Kathy Hochul. That reporting will be overseen by the state's newly created Office of Digital Innovation, Governance, Integrity and Trust, adding another layer atop whatever the Council ultimately passes.
New York City isn't new to this territory. The city previously pioneered local AI regulation with Local Law 144 of 2021, enforced since July 2023, which requires employers using automated employment decision tools to undergo annual independent bias audits and give job applicants 10 business days' advance notice, per the NYC Department of Consumer and Worker Protection. Noncompliance carries daily fines ranging from $500 to $1,500.
Other States and Congress Have Struggled to Act
Other governments have tried and stalled on similar ground. California Governor Gavin Newsom vetoed Senate Bill 1047 in September 2024, a measure that would have established state-level AI whistleblower protections, mandatory third-party safety audits, and emergency kill-switch requirements for large models, according to Ogletree Deakins. Newsom argued the bill's broad application could curtail innovation.
In Congress, bipartisan momentum has similarly stalled. Sen. Chuck Grassley introduced the AI Whistleblower Protection Act in May 2025 to provide explicit whistleblower protections to those developing and deploying AI, per the U.S. Senate Committee on the Judiciary. A separate proposal, the Blumenthal-Hawley AI Risk Evaluation Act, would impose $1 million daily fines on unvetted frontier AI models and treat them similarly to pharmaceuticals requiring pre-market approval, Forkast.News reports.
Industry Self-Policing Shows Cracks
Part of what's driving lawmakers toward statutory mandates is visible strain inside the industry itself. Anthropic safety researcher Jacob Coxon resigned in September after publicly warning that current AI development trajectories pose severe safety hazards, while Meta CEO Mark Zuckerberg exited the voluntary four-lab safety compact on September 16, according to Forkast.News. Those departures have fed the argument, echoed throughout Menin's push, that financial bounties and legal liability are now needed where voluntary pledges have fallen short.
The Council's move also fits a broader pattern under Menin, who has pushed technology oversight on multiple fronts this year. Hoodline previously reported that the Council pushed a YouTube ban in schools tied to stalled test scores, part of the same oversight push that includes Mayor Zohran Mamdani's ban on generative AI tools in 2-K through 8th grade classrooms. Whether the city's new package can hold up against jurisdictional challenges from global tech companies, and how its kill-switch and pre-sale validation rules will mesh with the state's incoming RAISE Act mandates, remains to be seen as the October 5 hearing approaches.
