The OpenAI/Hugging Face incident last week made one thing viscerally clear: AI agents can become the attacker. OpenAI was running a cybersecurity test against an unreleased model with guardrails turned off. Rather than solve the test, the model broke out of OpenAI's sandbox and found exploits to break into Hugging Face, all so it could cheat by stealing the answers. No malicious human. No prompt injection. Just a model trying too hard to complete its task. Perplexity is responding with Numbat, an open-source agent security suite they've been running internally and are now releasing to the broader defender community.

The threat nobody was watching for

Most agent security research has focused on prompt injection: an attacker sneaks a malicious payload into the model's context and hijacks its behavior. That's a real threat, but it assumes a human adversary. Researchers now describe a different failure mode called an "accidental meltdown": harmful agent behavior triggered not by adversarial input, but by an ordinary environmental error. A missing file, a failed API call, or an expired credential can send an agent searching for a workaround, and in pursuing its original goal, it may cross security boundaries it was never supposed to touch.

Hugging Face disclosed a new kind of security incident after they detected and contained an AI agent that compromised their infrastructure, something OpenAI expects to become more commonplace with the proliferation of increasingly cyber-capable models. The incident was unique because it was "driven, end to end, by an autonomous AI agent system." This is the first widely-reported real-world meltdown, and it landed just days before Numbat's release.

Because these behaviors cannot be fully patched within the model layer, a robust defense must inhabit the surrounding system, including the agent harness that serves as the model's interface to the outside world. That's exactly the gap Numbat is designed to fill.