Skip to content
by InfinyAI ES

IA en un minutoNewsRegulation

Regulation

Newsom proposes letting California order an emergency shutdown of a dangerous AI model

IA en un minuto newsroom · Editor: Jon Elgezabal

In 30 seconds

Gavin Newsom wants California to have room to stop an AI system at once when there is imminent danger and the company in charge fails to act. California is weighing intervention in layers, from compute through to deployment, with containment plans reviewed by outside verifiers. CalMatters notes that none of this is law yet and that the Legislature can take it up from December 7.

On Wednesday, November 18, California Governor Gavin Newsom presented the AI Safety in 2026 and Beyond report, drawn up by the state's Government Operations Agency together with a group of four experts and the California Office of Emergency Services. One of its recommendations is to give the state the power to halt an AI model immediately, as an emergency measure, when there is credible reason to see an imminent threat and the developer will not contain it.

According to the report, this is not one on/off button, what some call a kill switch, but the capacity to step in at several layers: compute, agents, internal access and the deployment of the model. Every large frontier model developer would need a risk containment protocol, which an independent verification organization would keep testing on an ongoing basis.

On incidents, it proposes cutting the deadline for reporting most critical ones from 15 days to 48 hours, keeping 24 hours when a catastrophic risk is imminent, and requiring notice of any loss of control over a model even when nobody is killed or injured, a condition that the SB 53 law, signed in 2025, sets today. The other recommendations include mandatory safety standards, independent audits at every stage of a model and the option to restrict AI that can design its own successor without human involvement.

The expert group is made up of Jason Goldman, Gillian Hadfield, Alondra Nelson and Rob Reich. These are recommendations, not laws: according to CalMatters, lawmakers will be able to file new bills when the Legislature returns to Sacramento on December 7.

Why it matters · analysis and opinion

The underlying change is who decides. Until now, California law asked large developers to publish their safety frameworks and report certain incidents, but the final word on stopping a model remained theirs. Under these recommendations, the state would keep that decision for itself in extreme cases and bring in third parties to check that companies know how to stop their own systems. The obligations would fall on frontier labs, not on those who use their tools, although any shutdown would affect the services built on top. The main step is still missing: the Legislature has to turn this into law and set concrete thresholds. The governor is already asking Washington to take this framework as a starting point.

Official source: Government of California · Written with the help of AI: how we make the news

AI in your inbox, every day or every Friday

The stories that matter, each one in a minute. With the source for every one.

Choose one or both:

Sign up and you are in: the daily arrives every night and the weekly on Friday mornings. You can unsubscribe from any email.