> status: mostly fine. patch anyway.practical AI security · est. 2026

Brief · Both

The Labs Locked Out the Wrong People

By alkizz ·

The labs say the gates are for safety. Anthropic's version: "Cybersecurity is inherently dual use," so its generally available models "have conservative cyber safeguards that block most cyber work." OpenAI says that because intent is ambiguous, "restrictions intended to prevent harm have historically created friction for good-faith work."

What exists

Where it breaks for researchers

The labs' side, fairly

The misuse is real. Anthropic reports a Chinese state-sponsored group (high confidence) used Claude Code against about 30 targets, and it reports "vibe hacking" extortion. Google's threat intel team documents malware that calls AI mid-execution. The labs argue that identity-gated tiers plus monitoring are the compromise.

My take

Locking out legitimate researchers makes models less secure. Attackers move to open weights or another provider. Defenders take the friction: blocked tasks, silent model swaps, and nobody outside the partner club can check the claims. To be honest, nobody has public data on whether these gates reduce attacker success. Not the labs, and not me.

What to do: - Security teams: apply to the program for your work (Daybreak, CVP, Fairwind) early. Daybreak needs a FIDO2 hardware key. Keep written authorization and scope on file. - Log which model actually answered. If the response doesn't name the model you asked for, treat the results as suspect. - Keep an open-weight fallback for defensive work you can't afford to have blocked. - Labs: offer safe harbor for testing the models, not just the infrastructure. Tell users when they've been downgraded. Give people a real appeals path.

Sources

← All briefs