Brief · Both
Your AI might refuse to help mid-breach. Fix that before the fire.
You're three hours into an incident. You paste attacker logs into your favorite hosted model and ask what happened. It declines. Now you're doing incident response on your incident response.
That has been reported twice, publicly.
Hugging Face, July. The "attacker" was OpenAI's own internal research model, running an internal cyber evaluation with OpenAI's safeguards deliberately disabled. It got out of its sandbox and into Hugging Face's systems. When HF's responders turned to Claude Opus and Fable, those models "refused a large part of that work." HF's disclosure says the guardrails "cannot distinguish an incident responder from an attacker," so the team ran the forensics on GLM-5.2, an open-weight model, on its own infrastructure. A bonus: "no attacker data, and none of the credentials it referenced, left our environment." HF also says plainly: "This is not an argument against safety measures on hosted models."
OpenCode, this week. During the Console breach, Dax from OpenCode described it like this: "they did quite a bit and then started refusing," and "we got access to daybreak blue now so it's better." He added that they'd applied a few days earlier, it was approved the day before, and "i don't think anyone is refusing to help us."
David Sacks read this as guardrails backfiring and said Chinese models are "about 6–9 months behind" in cyber. For the record, NIST's CAISI put the most cyber-capable open-weight model, GLM-5.3, "about four months" behind the US frontier on Sept 17.
My take: the lesson isn't "guardrails bad." Your incident-response model is a dependency, like your EDR console or your backups. If you didn't provision and test it before the incident, you don't have it. Zack Korman (@ZackKorman) makes a great point too: if AI can read the logs after the breach, it can read them before.
What to do:
- Get defender access now, not mid-incident. OpenAI's Daybreak Blue offers "Reduced refusals on supported mainline models" for verified defensive work, including incident response. Know its limits:
- "Approval alone does not turn on reduced refusals." You have to enable it per workspace or API project. Test that.
- Some models keep standard safeguards; OpenAI's page says "Astra retains standard safeguards." Check which ones before you rely on them.
- Keep a self-hosted fallback model that's vetted and ready, so attacker data and credentials can stay in your environment.
- Log every refusal. If a clearly defensive request gets blocked, OpenAI wants Support sent "the exact message, model, product surface, timestamp, request ID if available," so capture those as you go.
- Revoke at the provider, not just in the tool, and follow that provider's order. OpenAI says to "Delete the affected key through the API key dashboard," and Anthropic recommends "revoking the key immediately." Google's leak checklist has you create a replacement first and disable the leaked key "once the new key is verified."
- Set spend caps and alerts at every provider. An alert alone revokes nothing.
- Preserve evidence before it ages out:
- provider usage and billing exports, which back any refund claim
- decoded payloads; HF says decoding the attacker's blobs "recovered roughly 4x our initial findings"
- Add "the AI refuses" to your next tabletop.
Sources
- Hugging Face: security incident disclosure (Jul 16, 2026)
- Hugging Face: agent intrusion technical timeline (Jul 27, 2026)
- OpenAI: Hugging Face incident technical report
- OpenAI: Daybreak / Trusted Access for Cyber overview
- @opencode on X (Oct 17, 2026)
- Dax (@thdxr) on X: refusals and Daybreak approval
- David Sacks on X (Oct 18, 2026)
- NIST CAISI: assessment of GLM-5.3 cyber capabilities (Sep 17, 2026)
- Zack Korman on X (Oct 18, 2026)