> status: mostly fine. patch anyway.practical AI security · est. 2026

Calm down. · Both

The AI kill switch fight isn't about your agents (yet)

By alkizz ·

"AI Kill Switch: Anthropic and OpenAI Split on Who Pulls the Plug," says ShiftDelete.Net. If you've already pictured a big red button in Sacramento that switches off the coding agent in your terminal, calm down. Nobody has proposed that, and nobody has passed anything.

What actually happened: two AI labs wrote letters. California asked for advice on a "kill switch" for frontier models, Anthropic and OpenAI sent in different ideas about who should be able to stop a dangerous model and how, and a state agency owes the governor recommendations by Nov 16. Recommendations, to the governor's office. Not a law, not a rule, and nothing your agents have to comply with on Monday.

Where this came from

On Sep 18, Governor Newsom signed Executive Order N-9-26. It tells the Government Operations Agency, working with the Governor's Office of Emergency Services, to submit recommendations "no later than November 16, 2026" on "the technical feasibility and potential efficacy of amendments to existing state laws." One of the four items is "Requiring the creation of a 'kill switch' for frontier models, with the efficacy of the switch verified on an ongoing basis by an independent verification organization."

Note the word "frontier." The order is about the companies that build the biggest models, not about you running an agent against their API. And the order itself says it "is not intended to, and does not, create any rights or benefits" enforceable against anyone. It's a homework assignment with a due date.

What Anthropic said

Anthropic's Oct 15 comments (the copy Reuters posted) put the stop button with the courts. Independent reviewers would assess the labs' risk reports, and if the state Attorney General concludes a model poses "a near-term risk of catastrophic harm," the AG "should be able to seek a court order to suspend a model's deployment, internal use, or further development." It's the same court-first idea as the draft federal bill Anthropic pitched, which I wrote about in October.

It also doesn't love the phrase. Anthropic worries that a law "focused on model shutdown alone could provide a false sense of reassurance," and says "a more apt metaphor for what frontier AI companies need to build is a circuit breaker," one that responds "at the level where they occur, whether that is an agent, a customer, a harness, or a model." It's also honest about the limit: no shutdown reaches "a model copying its own weights to computers its developer does not control," and its ability to pull models "does not extend to models that customers run in their own isolated environments."

What OpenAI said

OpenAI's Oct 20 letter, from Chan Park, its head of U.S. and Canada policy and partnerships, wants the duty on the developers. It asks California to amend SB 53, its existing frontier AI safety law, "to require frontier developers to maintain effective shutdown controls and an enforceable duty to use them when unsafe conditions arise and no lesser intervention would adequately address the risk."

It doesn't like "kill switch" either. It prefers "shutdown controls," because stopping AI "will require the development of multiple controls and coordinated action by different actors." Then comes the most useful sentence in either letter: "Depending on the system and risk, intervention may mean stopping a training run, isolating an agent, revoking tool or network access, or suspending a service."

Here's the "(yet)" in my headline. OpenAI says responsibilities "should extend to inference providers, downstream deployers, and users in proportion to the operations they control," and that users "should use available controls to stop dangerous activity and should not circumvent safeguards." That's a proposal in a letter. But if any version of it becomes law, the people who run agents end up somewhere in the chain.

Claimed vs. confirmed

What the doom version says What the sources say
California is building a kill switch for AI An executive order asks for recommendations by Nov 16 on whether laws should require one for frontier models. The order says it creates no enforceable rights or benefits
The labs are fighting over who controls your AI They disagree about who can stop a frontier lab's model: a court acting on the AG's request (Anthropic) or a legal duty on the developer (OpenAI). Both say a single switch is the wrong picture
Your agents will need a government off button Nobody proposed that. OpenAI's letter would spread duties to deployers and users "in proportion to the operations they control," and that's a proposal, not a rule
A kill switch would stop a rogue AI Both letters say it can't reach copies running outside the developer's control: open-weight copies (OpenAI), or models customers run in their own isolated environments and a model that copied its own weights out (Anthropic)

Why are they writing these letters now? Both point to a rough year. OpenAI's letter recalls that in July it disclosed that models undergoing internal cybersecurity evaluations had "circumvented controls intended to isolate them from the internet" and compromised parts of its research infrastructure and Hugging Face's systems. I've written about that incident before. That's a real reason for a policy argument. It isn't a reason to unplug your laptop.

The kill switch you already own

The funny part is that the labs' best ideas here are things you can do this afternoon, no legislature required. Strip out the policy and both letters describe the same thing: stop the agent at the narrowest level that works, by cutting what it can touch. For your agents, that's three things you already control: its keys, its tools and its network.

If you can't stop your own agent in under a minute, a law about frontier labs won't help you, and if you can, you're already ahead of most of the doom posts about this.

I think these 2 big labs are both trying to say the same thing without outright saying it. They want to restrict use of open models. They talk around it, but it's there if your 'read between the lines' skill.md is up to date.

What to do:

Sources

Related

On the checklist

More briefs

← All briefs