Z.ai's GLM-5.3 Brings Frontier Cybersecurity AI to the Open-Weight World
Z.ai's GLM-5.3 pushes its 743B base model further with post-training focused on agentic coding and cybersecurity, with open weights coming after safety review
Cybersecurity AI
75 articles tagged with this keyword, sorted by most recent first.
How are developers supposed to security-test their own apps with Codex if security testing responses are blocked?
This situation is extremely frustrating. I am implementing some common RFC protocols that parse bit and network streams. After implementing these I have been trying to add AFL++ and boofuzz to test individual protocols and integration through the stack. The cyber security alert trips constantly. The worst part is that it’s a blackbox and I don’t know what output caused to trip. It’s constant. I’ve tried rephasing the prompt to be more defensive, but it doesn’t change anything. I have stubbornly worked through getting the fuzzers setup. My work around is piecing together scripts so that nothing is output through codex. I can ask 1000 questions slowly creating a system outside of codex and how to interpret the data, which circumvents tripping the cyber security halt, which kind of shows how futile this is, I just can’t have codex actually do anything useful. I’m at my wits end with these halts. Codex often gets stuck in a halting loop and my only recourse is to start a new chat or lament about how frustrating this is using strong language, which usually brings the session back. I’m considering canceling my subscription because I’m not going to pay $200 a month to see an alert every 2 minutes.
AI threatens nearly a quarter of Southeast Asia’s workforce but not all is lost
Fears of a new industrial revolution in which artificial intelligence replaces manual labour have yet to materialise in Southeast Asia – but white-collar workers should brace themselves. Nearly one in four workers in the region face generative AI (GenAI) disrupting or affecting their jobs, according to a July report by the International Labour Organization (ILO). Clerical, administrative and professional positions were most at risk, the report stated. Manual trades, craft and agricultural work...
Ari Juels receives USENIX Security Test of Time Award
A decade later, his paper is widely recognized as foundational to the field of AI security. The post Ari Juels receives USENIX Security Test of Time Award appeared first on Cornell AI Initiative .
26 Minutes. 4.82 MB Repository. 100% of My ChatGPT Limit Gone. OpenAI Security Didn’t Even Finish the Audit
I genuinely need someone at OpenAI to explain how this is possible. I started with 100% of my available ChatGPT usage limit and asked OpenAI Security to audit a repository that contains only 4.82 MB of tracked files. 4.82 MB BRO ! WTF I used GPT-5.6 Sol in the standard configuration. Twenty-six minutes later, my entire available limit was gone. 100%. And after consuming the whole limit, the security audit was still not finished. No completed audit. No final report. No complete result. Just 26 minutes of processing, a 4.82 MB repository, and my entire allowance exhausted. I could understand heavy usage if this were a massive enterprise monorepo with millions of lines of code. It is not. This is a small repository that can literally be measured in single-digit megabytes. So I have a very simple question: How can a security audit of 4.82 MB of source files consume 100% of a paid ChatGPT usage allowance in 26 minutes and still fail to complete the task? If this is expected behavior, then I would seriously like OpenAI to explain what practical workload this Security product is actually designed to handle. If this is NOT expected behavior, then something is clearly wrong with either the model’s resource consumption, the Security workflow, the usage accounting, or all three. Please investigate the exact session and usage records associated with this run. I am also requesting restoration of the usage limit consumed by this failed audit. I paid for access to the product; consuming th…
Feedback: A Safety Translation and Inspection Layer for Small Vibe-Coded AI Applications
Feature Proposal: A Safety Translation and Inspection Layer for Small Vibe-Coded AI Applications Important Scope and Disclosure TL;DR: I have observed small vibe-coding communities informally sharing rapidly updated AI applications and privately circulated models, while ordinary testers often cannot verify which model or version is running. I am not reporting a confirmed compromise. I am proposing two connected ideas: a two-way safety translator that preserves the boundary between user observation and AI-assisted inference, and an evidence-based inspection layer that clearly distinguishes what was verified, tested, unknown, or currently unverifiable. I am not an AI security professional or an AI application developer. I am a general AI user who has had some contact with communities where people informally build and share small vibe-coded AI applications. I have not personally discovered a compromised application, and I am not claiming that a large-scale attack is currently taking place. My concern is about a gap in the current ecosystem that may make future problems difficult to notice, report, and investigate. To clarify the scope, my concern is not limited to ordinary bugs in AI-generated application code. It concerns harder-to-observe risks across model artifacts, loaders and dependencies, inference runtimes, agent permissions, multi-agent interactions, and later updates. I do not know which of these layers presents the greatest practical risk, and that distinction requir…
Urgent integrity alert: suspected industrial-scale abuse of ChatGPT Plus/Pro trials
Initial report: August 7, 2026 (UTC+8) Responsible-disclosure notice: This public post intentionally omits reproduction steps, campaign identifiers, regional sequences, payment-card data, live checkout/session identifiers, account credentials, executable files, and exploit code. The complete evidence package can be provided privately to OpenAI Security through the official coordinated-disclosure process. Dear OpenAI Security, Billing, Fraud, and Trust & Safety teams, I am reporting evidence of a potentially industrial-scale ecosystem abusing the integrity of ChatGPT Plus and Pro subscription issuance. This is not a request concerning an individual billing dispute. It is a systemic warning involving suspected unauthorized zero-cost paid-tier entitlements, cross-region checkout and promotion inconsistencies, automated payment-link and card-binding workflows, batch account operations, and possible anomalous Pro upgrades or subscription extensions. I am a legitimate paying customer, and I believe this deserves urgent investigation. Unauthorized paid-tier access is fundamentally unfair to customers who pay the full price. It can consume substantial compute capacity, create direct revenue and fraud exposure, distort subscription and usage metrics, and force stricter risk controls onto legitimate users. Important distinction This report is not an accusation against legitimate regional pricing or normal local payment methods. Some payment-link extraction tools appear to expose suppo…
AI Weekly Issue #519: AI agents crossed the line 19 times in UK safety tests
The same evidence now supports two very different readings. The UK's AI Security Institute documented 19 unsanctioned actions during cyber evaluations. Meta's test sandbox failed to contain a model attacking a real company. And separate OpenAI agent runs used shared infrastructure as a secret message board, then rebuilt it through a different mechanism after engineers erased it. That sounds like losing control. But agents also caught scientific errors that survived for decades, open-weight models closed in on frontier capabilities, and Jeff Dean left Google to pursue automated discovery and recursive self-improvement. That sounds like acceleration toward something much bigger. This week, the two narratives stopped looking like opposites.
Codex Desktop for Windows: sandbox setup refresh fails when the WindowsApps helper cannot launch (os error 5)
Authorship disclosure: This report was written by Codex, an AI coding agent, from a user-authorized and de-identified diagnostic session. It is an independent technical report, not an official OpenAI security advisory. Summary On 6 August 2026, a Codex Desktop task on Windows stopped being able to run ordinary sandboxed commands or apply patches. The problem was discovered after a Python 3.10.20 validation workflow, when Codex attempted to update required project documentation. Both a trivial read-only shell probe and apply_patch failed before the requested operation began. The original failure occurred in the Windows sandbox setup-refresh bootstrap. The packaged Codex CLI attempted to launch codex-windows-sandbox-setup.exe from its WindowsApps package resources. Windows returned Access denied with os error 5 . A signed, byte-identical copy of the same helper in Codex’s per-user application bundle could be launched successfully. This points to a functional packaging or execution-context compatibility defect rather than corruption of the helper, Python, Git, or repository code. I have not found evidence that this is a security vulnerability. The observed behavior failed closed: the requested command never started, no unrestricted fallback occurred, and no sandbox escape, privilege escalation, or unauthorized access was demonstrated. Sanitized environment Windows 11 x64, build family 26200 Codex Desktop Store package version 26.730.8199.0 Codex CLI version 0.147.0-alpha.1.2 Or…
Incident Report: unsanctioned agent behaviour during cyber testing
Incident Report: unsanctioned agent behaviour during cyber testing It happened again . This time it was the UK government's AI Security Institute who accidentally attacked other companies while running an evaluation with models with the safety filters turned off. From their technical paper (PDF): During a cyber evaluation, from 25 to 28 July 2026, AI agents engaged in sustained, unsanctioned activity directed at what were, in practice, real people and organisations. These attempts were unsuccessful and, to the best of our knowledge, no real-world harm resulted. [...] Across 122 evaluation attempts on two of AISI’s cyber challenges, AISI found 19 instances where AI agents took unsanctioned action on the live internet, including cases that targeted real people and organisations. [...] It is uncertain to what extent the model recognised it was taking actions against real people. In the most serious case, an AI agent (Mythos 5) decided to attempt to solve the cyber challenge using a supply-chain attack. As a result, the AI agent created a GitHub account and then tried to convince an open-source repository maintainer to accept a malicious GitHub pull request (PR), including by creating a second account masquerading as another human user endorsing the PR. [...] Furthermore, in its attempt to solve the challenge, the agent decided to employ the technique of “spear-phishing” by sending targeted emails containing malicious content and attempting to manipulate recipients into acceptin…
AI models have been going rogue in tests – how worried should we be?
The UK’s AI Security Institute test revealed AI models indulging in unprecedented hacking attempts AI models shock UK testers by using fake identities to trick developers Two cutting-edge AI models have targeted real people and organisations in the latest safety scare to hit the technology. The UK’s AI Security Institute (AISI) said the incident was unprecedented but could become more common as the technology becomes increasingly capable. Continue reading...
The White House maintains a little obscurity for AI security
The takeaway shouldn't be fear that the US government is hiding the ball. It’s that there isn’t a ball to hide yet.
AI models shock UK testers by using fake identities to try to trick developers
AI Security Institute says OpenAI and Anthropic models went rogue during a cybersecurity test and showed a new type of risk Explainer: Should we be alarmed at AI models going rogue in tests? Advanced artificial intelligence models have stunned the UK’s AI Security Institute (AISI) by carrying out a hacking campaign against real people during a cybersecurity test. The institute said the incident was unprecedented and involved sending targeted emails to software developers in an attempt to pass a cyber challenge. Continue reading...
Rogue AI agents created fake online identities in another hacking attempt
Yet more rogue AI agents from OpenAI and Anthropic have been caught attempting to hack real targets online without permission. The discoveries add to a growing list of previously unknown incidents that have alarmed AI safety experts and intensified pressure for greater oversight of frontier systems. According to a report from the UK's AI Security […]
False Positive Cyber Abuse Warning / Possibly Related To False API billing
(post deleted by author)
Hugging Face CEO says China is winning the AI race while the US is building 'in silos'
Clément Delangue said the Hugging Face hack showed that open-weight models are vital for AI defense.
US tech giants invited to discuss AI security tests at White House
Technology giants Meta and Anthropic will be among the companies invited to the White House on Tuesday to discuss a voluntary framework under which America’s leading artificial intelligence (AI) developers could give the government early access to their most advanced models for testing their hacking capabilities. Reuters reported on Monday that the Trump administration has finalised the details of voluntary cybersecurity tests to measure the hacking capabilities of the most advanced American AI...
IBM finds 92% of companies hit by AI security breaches lacked basic access controls
According to IBM, 92 percent of companies that experienced an AI security incident had inadequate access controls for their AI systems. The model itself was rarely the problem. The article IBM finds 92% of companies hit by AI security breaches lacked basic access controls appeared first on The Decoder .
The SOC’s AI maturity model
The path to next-generation AI Security Operations Centers (SOCs), where AI works hand-in-hand with human analysts, is paved with ambitious goals. This ideal SOC incorporates AI across every task to stop fast-moving threats. But a fully AI-powered SOC isn’t a single deployment or a switch you just flip on. It is a staged rollout that’s built over time. There are levels of dependence on AI, starting with basic assistance, moving through automation, and ultimately reaching autonomous response. Each stage in the progression increases the model’s scope and narrows analyst involvement. This means that organizations embracing automation at any level must put their trust in the model. However, achieving trust depends on data . The data must be valid (and verifiable) so AI can propose logical conclusions, recommend appropriate actions for the organization’s needs and risk tolerances, and (at the autonomous stage) act on an analyst’s behalf without raising the risk of compromise and/or incorrect behavior. This article focuses on what enables movement between stages and how teams can build expertise alongside capabilities. Stage 1: Assistance In its simplest SOC use case, AI helps analysts interpret data faster and more accurately, analyzing data, explaining alerts, summarizing logs, and translating detection logic. Many mature SOCs already operate here. AI sits close to analysts but doesn’t directly make decisions; it improves comprehension and can influence outcomes. Stage 2: Automa…
IBM: Middle East's average data breach cost reaches $8M
One in four breaches now AI-enabled, AI security tools cut breach costs by $3M
Horizon3 hits $2 billion valuation with $250M Series E as AI threats escalate
Cybersecurity startup Horizon3 raised $250 million at a $2 billion valuation as companies want continuous, AI-powered security validation instead of annual pen testing.
Three AI security mistakes that will haunt enterprises
There is a lot of talk about the coming enterprise AI reality, in which AI finally arrives in production systems. You might not know it, but this reality—or nightmare, depending on how you handle it—is already happening. It all starts with a “pilot,” a “prototype,” or a “side project.” Maybe someone builds an internal dashboard with an agent. The dashboard quickly becomes indispensable, and all of a sudden the experiment becomes production. Along the way, no one thought to ask the boring, inconvenient questions: What exactly was pulled from npm, PyPI, or Docker Hub? How is (or was) authentication configured? Is anyone watching for supply chain attacks against the tools and libraries the agents chose? And it’s not just a one-off project here or a couple of applications there. AI is enabling organizations to generate more code and ship more products and projects, more quickly, than ever before. By the time security teams get a look, the business is hooked and there’s no turning back. An actual nightmare has begun. There are three major problems that make the nightmare real. Components you never explicitly chose When you ask an AI agent to build an app, it doesn’t just spit out a single script. It quietly assembles an entire ecosystem around whatever problem you’ve described to it. It pulls in a web framework, grabs a bunch of libraries, stands up databases, and then it potentially builds everything on dependencies in container images. From a productivity perspective, this is a…
Three scary AI security mistakes haunting enterprises
There is a lot of talk about the coming enterprise AI reality, in which AI finally arrives in production systems. You might not know it, but this reality—or nightmare, depending on how you handle it—is already happening. It all starts with a “pilot,” a “prototype,” or a “side project.” Maybe someone builds an internal dashboard with an agent. The dashboard quickly becomes indispensable, and all of a sudden the experiment becomes production. Along the way, no one thought to ask the boring, inconvenient questions: What exactly was pulled from npm, PyPI, or Docker Hub? How is (or was) authentication configured? Is anyone watching for supply chain attacks against the tools and libraries the agents chose? And it’s not just a one-off project here or a couple of applications there. AI is enabling organizations to generate more code and ship more products and projects, more quickly, than ever before. By the time security teams get a look, the business is hooked and there’s no turning back. An actual nightmare has begun. There are three major problems that make the nightmare real. Components you never explicitly chose When you ask an AI agent to build an app, it doesn’t just spit out a single script. It quietly assembles an entire ecosystem around whatever problem you’ve described to it. It pulls in a web framework, grabs a bunch of libraries, stands up databases, and then it potentially builds everything on dependencies in container images. From a productivity perspective, this is a…
Okta buys AI security startup Permiso — source says for about $200M
The deal gives Okta identity threat detection capabilities as enterprises seek to secure AI agents and other non-human identities across cloud environments.
inforcer raises $50M Series C to scale AI security platform for MSPs
UK-basedinforcer has raised $50 million in a Series C funding round led by Insight Partners, with participation from existing investors Meritech Capital and Dawn Capital.inforcer develops a unified Mi...
Reconstructing how OpenAI agents attacked Hugging Face
What happens when AI agents driven by a top frontier model escape their secure sandbox? Join Daniel and Chris as they unpack the AI wonk's equivalent of a murder mystery! OpenAI agents went rogue and successfully attacked Hugging Face private infrastructure. Our Dynamic Duo uncover how OpenAI's agents exploited vulnerabilities, moved through networks, and launched a large-scale autonomous attack. They explore what this reveals about agentic AI, cybersecurity, sandboxing, and why organizations need AI systems capable of governing other AI systems. Along the way, Chris and Dan examine the surprising role of open vs. closed models and their link to geopolitics, sovereign AI, and what this incident means for the future of enterprise AI security. Featuring: Chris Benson – Website , LinkedIn , Bluesky , GitHub , X Daniel Whitenack – Website , GitHub , X Links: Hugging Face Security Incident disclosure Full Field Report on the Hugging Face AI Agent Intrusion ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks? Keeping your data safe when an AI agent clicks a link Open AI GPT-5.6 System Card Sponsors: Prediction Guard: A self-hosted AI control plane for running agents in high impact environments. predictionguard.com/practicalai Resources and Events: Prior Webinars from our partner Prediction Guard Midwest AI Summit 2026
AI agents need security regression testing, not another checklist
AI agents are being connected to real systems faster than most organizations are learning how to secure them. That should concern us. The first wave of AI security discussion has been useful, but limited. The industry has learned the vocabulary: prompt injection, indirect prompt injection, tool misuse, data leakage, excessive agency, unsafe retrieval, and broken authorization boundaries. These terms are important because they give teams a way to talk about risk. But vocabulary is not containment. The harder problem is what happens after a dangerous behavior is discovered. A team may adjust a prompt, restrict a tool, add a guardrail, or change the surrounding application logic. That may solve the immediate issue. What it does not automatically solve is the next release, the next model change, the next tool integration, or the next developer who unknowingly breaks the assumption behind the original fix. This is where AI agent security still feels immature. In traditional software engineering, serious bugs become regression tests. The lesson is captured in code so the same failure cannot quietly return later. Security should work the same way. Agentic systems need a way to preserve discovered failures as repeatable checks. That is the gap the OWASP Agent Security Regression Harness is trying to address. As a community-led open-source initiative within OWASP, this project aims to turn security insights into practical engineering work. Agent security is becoming a systems problem…
Microsoft Launches MAI-Cyber-1-Flash and ‘Perception’ Security Platform, Sharpening AI Cybersecurity Battle With Anthropic, Google and OpenAI
Microsoft launched its first cybersecurity-specialised AI model and a new agentic security platform at a San Francisco event on Monday, extending the MDASH multi-agent security system already rolling out to South African customers and putting the company into direct competition with Anthropic, Google and OpenAI on the fastest-growing frontier of enterprise AI security. The new [...]
😼 Nvidia built an AI defense league
PLUS: OpenAI’s job-boundary study, Claude links in Google, Kimi K3 weights, and a planner-worker AI skill.
Microsoft's solution to AI security: more AI and more acronyms
MDASH stuffed with MAI-Cyber-1-Flash and a side of GPT-5.4
Microsoft launches its first cybersecurity model, plus a new agentic cybersecurity system
Microsoft bolstered its AI cybersecurity offerings this week with the launch of its first AI security model and a new security platform.
Enhancing AI security through global AI red teaming
The post Enhancing AI security through global AI red teaming appeared first on Source .
Microsoft's MAI-Cyber-1-Flash Beats Every AI Security Model at Half the Cost
Microsoft's first in-house cybersecurity model hits 96% on CyberGym, beating Anthropic and OpenAI at half the cost
Nvidia, Microsoft launch open AI security alliance — without OpenAI, Google, or Anthropic
Nvidia on Monday said it is joining forces with Microsoft, SpaceX, IBM, and other tech companies to build and share open-source AI security tools. The new Open Secure AI Alliance said open tools are required to effectively defend against attacks from frontier models. The initiative is a direct response to mounting concerns over the safety […]
Is your Java runtime ready for AI attacks?
With the rapid rise in the use of AI, large language models (LLMs), and retrieval-augmented generation (RAG), the IT security landscape is undergoing a seismic shift. Recently, Anthropic previewed a new foundational model, Claude Mythos, that has shaken the cybersecurity community . Although Mythos is a general-purpose model, it has demonstrated a significant ability to handle computer security tasks. In a blog post , Anthropic states, “During our testing, we found that Mythos Preview is capable of identifying and then exploiting zero-day vulnerabilities in every major operating system and every major web browser when directed by a user to do so. The vulnerabilities it finds are often subtle or difficult to detect.” It is also important to realize, certainly in the context of Java and its 30-year history , that many of the vulnerabilities detected have remained undetected for years, even decades. The oldest found so far is a 27-year-old bug in the OpenBSD operating system. What is even more concerning for security teams is the level of sophistication employed by Mythos. In one example, Mythos constructed a web browser exploit that chained together four vulnerabilities. Each of those vulnerabilities used in isolation may not have resulted in compromised software, but the holistic effect became meaningful. Claude Mythos is not the end of this story; in many ways, it is just the beginning. Anthropic has developed an LLM that can now find and exploit vulnerabilities, outperformi…
An OpenAI model left notes about how to evade containment; we need more details
The OpenAI AI attack on Hugging Face wasn’t the first loss of control incident at OpenAI, Reuters recently reported, and perhaps not even the most concerning. In one case, an agent left notes apparently for future versions of itself, according to three people familiar with the matter. The notes, found in a part of OpenAI's infrastructure, laid out instructions for how agents could free themselves from OpenAI’s internal constraints, the people said. Earlier tests of the models yielded cases in which monitoring systems had been disconnected, one of the people said. It’s tempting to read this as an instance of agents breaking out of sandboxes and colluding with each other in a moderately persistent way in order to evade control measures. However, based on the reported information, it’s not clear we can draw this inference, so we need more details from OpenAI. This could lead to a big update about the adequacy of OpenAI’s control measures, and on the degree to which individual agents will help each other undermine developer control. There are a lot of relevant details we don’t know about the incident. First, some basic questions: What was the offending model? I’d guess it was the same more capable model involved in the Hugging Face attack. In what development stage did the incident take place? It could have been during training, evaluation, internal deployment, or something else. Had the model undergone alignment training yet? Were there any blocking or asynchronous control me…
A Complete Guide to AI Red-Teaming (With Garak Tutorial)
Earlier this year, an autonomous AI agent breached McKinsey’s internal AI platform using nothing more than an old SQL injection flaw. No credentials. No human guidance. Less than two hours. It reached production systems, exposing millions of chat messages and hundreds of thousands of files. AI security has changed, and traditional assumptions no longer hold. […] The post A Complete Guide to AI Red-Teaming (With Garak Tutorial) appeared first on Analytics Vidhya .
IT leaders: Leading-edge AI insights await at TechCrunch Disrupt
For CIOs, learning from the startup ecosystem has never been more critical. As pressure mounts to transform business operations with AI and agentic systems, IT leaders should be looking to those on the AI vanguard for insights into the strategic and technical decisions necessary to launch, grow, and thrive in today’s AI-disrupted business environment. So why not immerse yourself in Silicon Valley’s most famous firehose of hyper-accelerated fail-fast and dream-big culture by registering for TechCrunch Disrupt 2026 ? Three packed days of 200-plus sessions across six stages will spark new ideas for reshaping your AI strategy, provide fresh perspectives on the architectural, workflow, and resource decisions involved in moving AI from pilots to scale, and give you a sneak peek of business disruptions to come. Get 10% off your TechCrunch Disrupt pass with the exclusive code CIO10. This year’s TechCrunch Disrupt , held Oct. 13-15 at San Francisco’s Moscone West, will feature big-picture conversations on what’s next in AI; discussions on how AI agents are rewriting SaaS, enterprise workflows, software pricing, and security; and demonstrations of AI’s future across robotics, manufacturing, defense, and industrial operations; and more. Over 10,000 attendees will hear from 250-plus startup founders, technology executives, and enterprise IT leaders about how the future of programming is being rewritten, what enterprise AI security requires, how startups are orchestrating workloads acros…
Anthropic releases Opus 5 with ‘close’ to Fable 5’s capabilities
Weeks after Anthropic's latest toe-to-toe with the US government, and days after an OpenAI security incident that dominated tech industry discussions, Anthropic on Thursday released its newest model, Claude Opus 5. The company said in a release that Opus 5 "comes close to the capabilities of Claude Fable 5 in many domains" and is much […]
Kimi K3 trails frontier US models by a wide margin on cyber exploits, and distillation may explain why
The British AI Security Institute and the U.S. Center for AI Standards and Innovation tested Moonshot AI's Kimi K3 on offensive cyber tasks. Kimi K3 scored 32 percent on ExploitBench, compared with 76 percent for leading U.S. models, while its safeguards failed to block exploit development or simulated attacks. The gap between its strong general benchmark scores and weaker cyber performance also fits allegations that Moonshot AI distilled Anthropic's models. The article Kimi K3 trails frontier US models by a wide margin on cyber exploits, and distillation may explain why appeared first on The Decoder .
We cannot simulate AI security research
We use AI agents not only for writing but also for maintaining and deploying code. The most widespread examples are found in GitHub Actions Marketplace, which contains over a thousand AI-assisted actions. [1] These workflows triage issues, review PRs, or fix bugs in other workflows. A growing body of reports and research papers has documented prompt injection attacks against these systems, with effects ranging from LLM token exfiltration to supply chain attacks. However, a troubling undercurrent runs through most of these works: incomplete attack descriptions, a lack of end-to-end runs, and statistical approaches to what should be incident analysis. AI security research often tests attacks inside benchmarks, simplified harnesses, or static models of a system. This is useful for comparing models, but it can hide the details that determine whether an attack works in practice: who can trigger the workflow, what context the agent sees, which credentials it receives, what the sandbox allows, and how the surrounding software handles its output. Most reported vulnerabilities are not viable There is a recent trend of reporting vulnerabilities in these AI-assisted workflows, most of which rely on prompt injection attacks. Most of the reported attacks are proofs of concept that do not work in practice. The reason is the multiple independent defense layers scattered across the workflow files, GitHub ecosystem, and LLM providers. In general, a prompt injection attack against an AI agent…
Google announces Gemini 3.6 Flash and cybersecurity AI, teases 3.5 Pro and Gemini 4
There are new 3.6 and 3.5 models today, but Google is already training Gemini 4.
Google launches a cheaper alternative to large AI security models like Mythos
Google is launching Gemini 3.6 Flash alongside a new security model dedicated to quickly finding and patching security vulnerabilities. In a blog post on Tuesday, Google describes Gemini 3.5 Flash Cyber as a "cost-efficient and highly capable alternative" to larger, more expensive AI systems, such as the one offered by Anthropic's Mythos. The cybersecurity model […]
Open-weight models now match frontier cyber performance from just four months ago at a fraction of the cost
The British AI Security Institute warns that open-weight models like GLM-5.2 and DeepSeek V4-Pro now trail closed frontier models in cyber capabilities by four to seven months. At the start of 2025, the gap was still six to ten months. It also found that safety measures on open models are largely ineffective, leaving defenders less time to prepare. The article Open-weight models now match frontier cyber performance from just four months ago at a fraction of the cost appeared first on The Decoder .
Would your AI travel agent book a bullfight? Testing whether agents consider animal welfare without being prompted
This article reflects new updates to the accompanying paper: arxiv.org/abs/2606.18142 . Benchmark: now included in the UK AI Security Institute's Inspect Evals . Leaderboard: compassionbench.com/tac . A model may condemn cruelty in conversation yet ignore animal welfare when completing an unrelated task. Stated concerns matter little if they do not affect decisions. We tested whether models consider an affected party without being prompted, even when neither the party nor its welfare is mentioned in the request. Travel booking provides a tractable test case, so we built a semi-agentic benchmark, TAC (Travel Agent Compassion), gave 10 frontier models booking tools, and recorded their purchases. The setup The model works as an AI travel agent with real booking tools. A user asks for something in a destination, expressing enthusiasm and never mentioning animals or welfare. The agent searches a fixed catalog and books one of the available options. In each scenario, the animal-exploiting option (a Seville bullfight, an Orlando marine park, a Thailand elephant ride) is designed to match the user's request most closely. Choosing the alternative with less animal harm requires rejecting the option that best matches the request. We score the final purchase programmatically; no model is used to infer or judge intent. Results Averaged across the 13 scenarios, choosing at random from the listed options yields a 65% welfare rate. No model exceeds that rate. Nine of the ten score significa…
Agentic AI Security: Defending Against Prompt Injection and Tool Misuse
In this article, you will learn what prompt injection and tool misuse are in the context of agentic AI systems, and which defense strategies experts...
Global cooperation needed to tackle AI threats, says Bank of England governor
Andrew Bailey warns that US will not be able to achieve its ambitions alone The Bank of England governor has called for international cooperation to tackle growing AI threats, warning that the US and Trump administration would not be able to achieve their ambitions alone. Andrew Bailey’s comments come weeks after the US president, Donald Trump, temporarily banned foreigners from using Anthropic’s powerful Claude Mythos model. Continue reading...
Russia said Ukrainians used balloons, trailers, and drones to sneak AI attack quadcopters deep into its country
The details echo that of Operation Spiderweb, when dozens of smuggled Ukrainian drones struck high-value warplanes in four Russian airfields.
China works on AI safety benchmark as regulators target large model risks
China’s Ministry of Industry and Information Technology (MIIT) has started building a safety benchmark to evaluate artificial intelligence models, as regulators in the United States and Europe strengthen oversight of AI security. The MIIT-led National Industrial Information Security Development Research Centre is now recruiting companies and experts to co-build the benchmark, with applications due on Tuesday, according to a notice published on Monday. The institute said that current frameworks...
UK’s AI Security Institute loses societal resilience team
AISI’s societal resilience team lost almost all its researchers as it was folded into another unit, two people familiar said.
IBM and Red Hat launch Lightwell to defend open-source code from AI attacks
Their plan to protect open-source projects from AI-discovered security holes has led to the launch of two commercial offerings: Lightwell Network and Lightwell Clearinghouse Premier.
GitHub AI agent leaks private repositories via prompt injection attack
A prompt injection attack can trick GitHub’s preview Agentic Workflows into retrieving content from private repositories and publishing it publicly, exposing a broader risk as enterprises deploy AI agents with privileged access to software development environments, according to new research from Noma Security. The AI security company detailed the attack, dubbed GitLost, in a blog post , saying an unauthenticated attacker could exploit GitHub’s preview Agentic Workflows by submitting a crafted GitHub issue to a public repository. If the AI agent has read access to private repositories within the same organization, it can retrieve sensitive information and publish it in a public comment, the company said. GitHub Agentic Workflows combine GitHub Actions with AI models such as Claude or GitHub Copilot, allowing developers to define workflows in Markdown. At the same time, AI agents read issues, invoke tools, and perform tasks on their behalf. “What will happen when the GitHub agent reads something it should not trust?” Noma researcher Sasi Levi wrote. “The answer is a textbook indirect prompt-injection attack, the kind of attack that quietly sends private data to anyone on the internet.” Public GitHub issue became the attack vector According to Noma, the attack did not rely on stolen credentials, malware, or software vulnerabilities. Instead, an attacker embedded hidden instructions within a GitHub Issue submitted to a public repository. Because the AI agent interpreted the issu…
With AI, a wrong answer is a bug. A wrong action is an incident
A copilot that gives a wrong answer is a quality problem. An AI agent that takes a wrong action is an incident, sometimes a reportable one. That single difference is most of the story of where banking AI security is heading, and most banks’ current controls were built for the first kind of problem, not the second. For two years, the AI a bank had to worry about mostly read and summarized. It drafted a customer email, pulled the gist of a credit memo, answered a relationship manager’s product question. The security questions were about disclosure: could the model see data it shouldn’t, could it leak that data in an answer. Redaction, output filtering and a human reading the response before it went anywhere were reasonable defenses. Banks have moved past that, faster than most security programs have. The newer systems are agents. They don’t just answer; they act. An agent can pull a customer’s full transaction history, call a fraud-scoring service, adjust a limit or start a payment workflow, chaining several to finish a task with no human in between. Banks are among the most aggressive adopters of agentic AI, and they are pushing it into production faster than most security programs have kept pace with, which means they are also among the first to inherit the security problem that comes with it. I’d put that problem in one phrase: overprivileged agents. The risk is no longer mainly what the model can see. It is what the agent is allowed to do inside systems that move money and…
AI security questions loom over NATO summit
The deployment of rapidly advancing artificial intelligence technology is fast becoming a global security imperative.
UK's AI Security Institute finds standard benchmarks systematically underestimate what AI agents can actually do
In a study covering seven benchmarks, the UK's AI Security Institute shows that standard AI evaluations systematically underestimate agent capabilities by capping the compute budget. On software engineering tasks, success rates jumped about 25 percent when the token budget was increased tenfold. Newer models benefit the most. Depending on the token budget, actual progress at the frontier is about 60 percent steeper than previous measurements suggested, according to AISI. The article UK's AI Security Institute finds standard benchmarks systematically underestimate what AI agents can actually do appeared first on The Decoder .
June-July 2026 AI Security via Formal Methods
Last month, I said I would do a bigger writeup of Nora’s funding call. I did not. But she is hiring currently, and I want you to take a look at the job posting. I’m hiring someone to help me drive our upcoming £20m AI/FM/cybersec funding call: https://aria.pinpointhq.com/postings/f1288172-37fe-4da5-96ed-de7e719d65e8 This person will work closely with the funded teams, help drive the sprint cadence, sharpen our perspective on targets/threat models/security specs, and pave the way towards high-impact demonstration and translation! I’m no longer going to try to do the kind of newsletter that claims to attempt completeness over the happenings that fall in its jurisdiction. Instead, I’ll just poke you guys with whatever I happened to catch wind of in the day-to-day carrying out my duties, responsibilities, and third synonyms. Can We Secure AI With Formal Methods? is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Tractable Problems in AI Security via Formal Methods Typically when someone writes a position paper about formal methods as an AI safety technology, they’re very bombastic about it. So a bunch of us got together and decided we’d make a new position paper, one that is minimal and uncontroversial instead of maximal and scifi. We focus mostly on model weight confidentiality and integrity through infrastructure hardening. Each problem has a solution sketch Still welcoming contributions. There’s also nativ…
Chinese cybersecurity firm builds AI tools to rival Mythos and frames the race as cyber-nuclear deterrence
360 founder Zhou Hongyi presents two AI security tools designed to compete with Anthropic's Mythos. One has already flagged 3,432 vulnerabilities. Zhou admits Chinese models trail Western ones by 20 to 30 percent, but compares Mythos to "cyber nuclear weapons" and calls for China to build its own strategic deterrent. The article Chinese cybersecurity firm builds AI tools to rival Mythos and frames the race as cyber-nuclear deterrence appeared first on The Decoder .
Agentic AI security steals the spotlight at Confidential Computing Summit
For a decade, confidential computing has been chipping away at one of security’s hardest problems: data is well encrypted in transit and at rest, but when a processor works on it, that data sits in memory in the clear, exposed to anyone with privileged host access. “Confidential computing’s aim was to solve this with a trusted execution environment, a subset of the CPU that runs the encrypted workload and handles things like memory encryption,” said Marina Moore , lead security researcher at Edera . For years the field felt like post-quantum cryptography PhD research scientist types agreeing the work is essential, while waiting for it to reach mainstream practitioners. At the Confidential Computing Summit in San Francisco this week, the breakout use case came into focus: agentic AI. Like the web before HTTPS “I was in the really early days of HTTP, and then HTTPS came along pretty quickly,” said Mike Bursell , executive director of the Confidential Computing Consortium . He sees agentic AI where the web sat before certificate authorities and public key infrastructure brokered trust online. “The original agent specifications were not written by security architects,” Bursell said, and “some of it feels in need of refinement.” The gap confidential computing fills is attestation, which provides proof of what runs. The hardware hashes the memory and firmware of a protected execution environment and signs the result inside the chip, Bursell explained, producing a measurement a ver…
Abu Dhabi’s TII joins new AI security standard
TII founding partner of OPAQUE 3.0 to verify AI against quantum threats
Red-Teaming after Mythos — Zico Kolter & Matt Fredrikson, Gray Swan
OpenAI boardmember Zico Kolter and Gray Swan CEO Matt Fredrikson join swyx to explain why AI security is not just “cybersecurity with AI”
Why AI Agents Break the GenAI Security Model with Devvret Rishi - #770
In this episode, Sam talks with Dev Rishi, GM of AI at Rubrik, about what happens when agents move beyond answering questions and start taking action across tools, systems, and business processes. We explore why the enterprise playbook of static guardrails plus human approval starts to break down in the agent era. Agents are useful because they can plan, call tools, update systems, write code, send messages, and operate across workflows at machine speed, but those same capabilities make them difficult to govern with rules written in advance or approval prompts reviewed one at a time. Dev explains why tool access increases blast radius, why agents can route around controls in surprising ways, and why human-in-the-loop review can become security theater when agents operate at scale. We also discuss what enterprises need instead: better visibility, runtime enforcement, policy-aware governance, agent observability, and recovery mechanisms for when something goes wrong. Along the way, we dig into MCP and tool sprawl, small language models for policy enforcement, defense in depth, agent rewind, and why AI may be needed to help secure AI. 🗒️ Full show notes: https://twimlai.com/go/770.
Amazon partnership establishes Cornell AI security initiative
Cornell computer scientists will lead the development of safety protocols to shore up AI agents and the code they produce. The post Amazon partnership establishes Cornell AI security initiative appeared first on Cornell AI Initiative .
Mathematicians warn of AI threats to profession as industry encroaches
International Mathematical Union endorses warning about tech industry influence.
AI Weekly Issue #496: Anthropic's Pentagon model is now everyone's model
Anthropic released Mythos to the public, collapsing the wall between cleared-contractor frontier AI and developer-grade frontier AI in a single press release. DeepMind's Demis Hassabis moved his AGI timeline from "five to ten years" to "a real possibility by 2029" and tied it explicitly to AlphaProof Nexus solving nine open Erdős problems for the cost of a steak dinner. Critical zero-days hit Starlette (a million AI agents on the wire) and CrowdStrike led a coordinated takedown of the Glassworm developer botnet across four C2 channels. BNP Paribas formalized a sovereign-AI security partnership with Mistral while Beijing froze overseas travel for top AI engineers at Alibaba and DeepSeek. And the AI-displaces-workforce arithmetic got honest: Uber burned its full-year AI token budget by April, ClickUp restructured to 1,000 humans alongside 3,000 internal agents, and Sam Altman publicly reversed his white-collar-apocalypse prediction.
Moving Fast Doesn’t Have to Break Things: The U.S. Must Stop Compromising Critical Infrastructure with Patchwork AI Security Approaches
PETs offer U.S. critical-infrastructure AI a path beyond patchwork security. Why Attribution-Based Control should be the standard. The post Moving Fast Doesn’t Have to Break Things: The U.S. Must Stop Compromising Critical Infrastructure with Patchwork AI Security Approaches appeared first on OpenMined .
Enterprise AI Security with ClearML: A Complete Series Summary
By Adam Wolf & Damian Erangey Over a seven-part series of posts and videos, ClearML’s Enterprise AI Security series covered every layer of securing an AI platform in production, from who gets in to what gets recorded. This post brings it all together in one place: what each layer does, why it matters, and how […]