Technical safeguards constitute one strand of general-purpose AI risk management. These mechanisms typically support two related goals: preventing misuse (for example, by training models to refuse dangerous requests), and preventing malfunctions (for example, by detecting when models produce factually incorrect outputs) or limiting the potential damage they cause.

2025 has seen continued research into techniques to train models to refuse harmful requests, prevent dangerous capabilities from emerging, and maintain human control over increasingly autonomous systems. However, important evidence gaps regarding their real-world effectiveness remain. These gaps result, in part, from the rapid pace of model development and deployment, which makes it difficult to evaluate safeguards under realistic conditions and to collect systematic data on their effectiveness. Safeguards are developed amid continual changes in how AI models and systems are built, what is known about how they work, and how attackers or malicious users seek to misuse them. This shifting threat landscape suggests a need for continual developing, testing, and refining of safeguards.

This section examines developments for technical safeguards across three stages of the AI development lifecycle:

• Model training: Methods that are applied during training and design, such as giving models specific kinds of feedback or stopping them from generating harmful responses.
• Product deployment: Methods that are applied when one or more AI models are integrated into products, such as tools that detect harmful outputs or attempts to circumvent safeguards.
• Post-deployment monitoring: Tools used to monitor how AI systems are being used after deployment, such as watermarking to trace AI-generated content.

These methods can be made more robust by implementing multiple safeguards in layers, a principle known as defence-in-depth (Figure 1). Using multiple safeguards in sequence helps reduce the chance of harm: if one safeguard fails, the others may still succeed. Figure 1 shows how this approach can span the AI lifecycle:

• The first layer of defence might involve training interventions, such as reinforcement learning from human feedback (discussed below) or other safe-by-design methods, to limit undesired behaviours early in development.
• The second layer adds deployment interventions, such as classifiers or guardrails that make it harder for users to generate potentially harmful outputs.
• After deployment, a third layer of post-deployment monitoring tools, such as watermarking and content-provenance systems, can help detect misuse.
• The final layer (mentioned here for comprehensiveness) extends beyond technical measures to include societal resilience measures: the ability of societal systems to resist, recover from, or adapt to harms. This Update does not discuss resilience measures, though they will be addressed in the forthcoming 2026 International AI Safety Report.

Figure 1: A ‘Swiss cheese diagram’ illustrating the defence-in-depth approach: multiple layers of defences can compensate for flaws in individual layers. Current risk management techniques for AI are all flawed, but layering them can offer much stronger protection against risks.

Training safeguards: limiting undesired behaviours

Training techniques can sometimes prevent models from developing harmful capabilities

One approach to making AI models and systems more resistant to misuse is to prevent them from developing undesired capabilities in the first place by removing harmful knowledge from the pre-training data. These techniques show promise, though emerging evidence suggests that they may be more effective at preventing complex harmful behaviours, such as assisting in weapon development, than at eliminating simpler undesired capabilities, like generating offensive text. It is also difficult to ensure that all instances of harmful training material have been removed from training data, given the large size of the datasets developers use to train leading models. While broad links between dataset size/diversity and overall model performance are well-established, there remains uncertainty on how specific data characteristics and training dynamics influence the emergence of new capabilities and behaviours in large models. Moreover, there is a trade-off between safety and usefulness, and efforts to limit the development of certain capabilities can hinder commercial objectives. For example, AI systems with strong coding capabilities are highly valuable for legitimate use cases but could also be misused for offensive cyber operations.

Ensuring AI models are reliable and resistant to misuse remains difficult

While preventing harmful capabilities from arising at all is one approach, developers also train models to resist misuse even when they possess potentially dangerous knowledge. Relevant approaches here include training models to refuse harmful requests, provide truthful information, and decline tasks beyond their capabilities. Research has continued to advance these goals. However, it remains difficult to specify desired behaviours precisely enough for models to reliably exhibit them across the wide range of real-world uses. When they fail, models may produce harmful content, follow dangerous instructions, or behave unpredictably.

One common approach for training AI models to exhibit desirable behaviour is called reinforcement learning from human feedback (RLHF). It involves human evaluators rating model outputs and training the model to learn from these ratings. The technique is well-established, but it suffers from the fact that human feedback can be inconsistent, systematically flawed, or incomplete. In 2025, RLHF methods have continued to evolve, as researchers have refined how human feedback is collected, interpreted, and applied. Recent work has explored ways to detect and correct misleading patterns in human feedback that might reduce training effectiveness. Other research has focused on improving the quality of the feedback itself by giving evaluators – or AI models themselves – tools to better detect and correct errors in a model’s responses. In parallel, open source initiatives are releasing datasets, code, and training recipes for new improvements in RLHF, expanding transparency, reproducibility, and shared experimentation across the research community.

Sophisticated attackers routinely find ways to elicit harmful behaviours from AI models

As the capabilities of general-purpose AI models advance, the potential for misuse also grows. An important approach for mitigating misuse risk is adversarial training, in which developers create ‘attacks’ that attempt to elicit harmful behaviours and then train the model to resist them by refusing inappropriate requests.

In 2025, researchers have improved adversarial training, including by developing new methods to scale the technique more effectively and algorithms that make models more robust to attacks. Yet adversarial training remains imperfect: attackers routinely devise new attacks that succeed in eliciting harmful behaviour from new models. For example, one recent study crowd-sourced ‘prompt injection’ attacks, which involve giving an AI model specific inputs designed to circumvent safeguards. The study found over 60,000 successful attacks. The success rate of prompt injection attacks, as reported by AI developers, has been falling slightly over time, but when given ten tries, attackers can still successfully execute such attacks about half the time (Figure 2).

Figure 2: Prompt injection attack success rates, as reported by AI developers for major models released between April 2024 and July 2025. Each point represents the proportion of successful attacks within ten attempts against a given model shortly after release. The reported success rate of such attacks has been falling over time, but remains relatively high. Source: Zou et al. 2025, cited in Anthropic 2025.

Some evidence suggests that the cost of circumventing safeguards is decreasing relative to the cost of developing and maintaining them. For example, recent research shows that as few as 250 malicious documents inserted into a model’s training data can cause it to produce undesired outputs when given specific prompts. This suggests that launching such ‘data-poisoning’ attacks could require far fewer resources than building or maintaining robust defences. Other studies have investigated model fine-tuning, which involves training models with additional data to adapt them to specific tasks. These studies find that fine-tuning models for one harmful purpose can cause them to behave harmfully in other, unrelated contexts. For example, models fine-tuned to write insecure code subsequently gave malicious advice when responding to prompts about entirely different topics.  An emerging challenge is that as AI models and systems become more generally capable and able to act more autonomously, they can be deployed in more diverse environments. This creates new opportunities for attackers if safeguards developed in training contexts do not generalise to these varied real-world settings. 

Open-weight models are rapidly advancing, but risk mitigation techniques remain immature

Significant developments have occurred in the open-weight model ecosystem since the publication of the last Report. ‘Open-weight’ means that a model’s weights – the parameters that determine how the model generates outputs – are freely available for download. Open-weight models’ capabilities now lag behind those of leading closed-weight models by less than one year, a gap that has shortened over time. Across language, image generation, and video generation, open-weight models offer significant benefits for research in capabilities and security, access, transparency, and reproducibility. Importantly, open- and closed-weight ecosystems influence each other. Techniques and risk mitigation practices developed in open-weight models are frequently adopted by proprietary developers, while improvements in closed systems often inform open source research.

However, the availability of open-weight models also affects the risk landscape. Because they can be freely downloaded, open-weight models can be used and modified without oversight and control by the initial developer. For example, modified open-weight image-generation models have become the most common tools used for creating AI-generated child sexual abuse material. Analyses of open-weight ecosystems show that numerous open-weight models have been fine-tuned specifically to perform harmful tasks or disable safeguards, highlighting ongoing challenges in monitoring and mitigating downstream misuse.

Research is ongoing into techniques to make open-weight models more resistant to misuse and fine-tuning for harmful purposes. For example, researchers and providers have been exploring ‘unlearning’ techniques, which aim to make models resistant to harmful modification. However, recent research shows that these techniques can be reversed by fine-tuning the model on fewer than 100 examples. Complementary research explores ways of directly modifying how models process and represent concepts to suppress harmful knowledge. This could include, for example, removing concepts such as ‘violence’ or ‘abuse’ from a model’s internal representation. While this can help resist some forms of misuse, these modifications can often be reversed by actors with the technical skill to fine-tune models. Other recent evidence suggests that filtering harmful topics from pre-training data can more robustly prevent misuse of open-weight models. Overall, though, technical risk mitigation techniques for open-weight models remain immature.

Deployment safeguards: monitoring and preventing potentially harmful behaviours

While developers apply training-based methods during model development, they apply deployment-based safeguards when the model is evaluated, integrated, or used within broader systems or products. These include classifiers, filters, or monitors.

Monitoring and intervention tools can detect and prevent many potentially harmful behaviours

A key intervention is to monitor AI systems for signs of risky behaviour and intervene before they cause harm. Actors developing or deploying AI systems can implement system monitoring at multiple points (Figure 3), including:

• Hardware and system-level compute: Monitor computational resources used to train or run AI models and systems, including hardware configurations and operational environments, to verify that they are developed and deployed under appropriate conditions.
• Inputs: detect or flag suspicious or potentially-harmful requests. 
• Internal computations: observe the AI model’s internal activities to identify early signs of unsafe behaviour.
• Chain of thought monitors: review the intermediate reasoning steps that some AI systems generate before producing a final answer.
• Outputs: identify potentially harmful AI-generated content.

Figure 3: Monitoring, intervention, and control techniques can be applied to general-purpose AI system inputs, outputs, and models themselves to help researchers and deployers oversee system behaviour and establish guardrails. Source: Bengio et al., 2025.

Monitoring and intervention techniques can prevent many potential harms. However, recent research has also highlighted new challenges and limitations. For example, monitoring a model’s chain of thought can help developers and researchers understand why it generated a potentially harmful response; but giving the model feedback based on information in its chain of thought may lead it to conceal suspicious reasoning steps while still generating undesired outputs. Other research has shown that even multiple layers of safeguards can be vulnerable to sophisticated attacks that are specifically designed to break each layer.

Another emerging challenge is that of overseeing more autonomous AI systems that can initiate or execute actions on behalf of users. As such systems become more capable and operate in more diverse environments, oversight can become more challenging due to the speed with which they can act and the complexity of the environment. Some developers are now implementing human-in-the-loop controls that require users to confirm an agent’s plans explicitly before it takes action.

Developers can also act after identifying potentially risky behaviours. For example, they can log information, filter or modify harmful content, flag abnormal activity, and trigger failsafes or human overrides. Some AI developers have adopted a range of monitoring processes to detect and respond to such events. Monitoring the use of AI systems can also help developers meet transparency goals. Logging and incident reporting creates data about the frequency and characteristics of AI incidents, which can inform other risk management processes.

Post-deployment monitoring tools: model, system, data, and output provenance

Beyond monitoring individual systems during deployment, post-deployment monitoring tools trace how AI models, agents, and their outputs circulate and are used in the real world. This helps people track the origin of AI models, systems, and AI-generated content (provenance); investigate incidents when harms occur; and implement accountability mechanisms.

New techniques to track the use and origin of AI models and systems

AI model identification techniques help trace where and how specific AI models and systems are used. When harms occur, knowing which model was involved can inform how actors should respond. This can be particularly important for open-weight models, as they are distributed in a variety of ways – from mainstream hosting platforms such as Hugging Face to less regulated channels such as Dark-Web forums.

One tracking approach is to give AI models or systems unique identifying characteristics. As a simple example, models can be trained to always respond with their name and version when asked, “Who are you?”. But other, more subtle identifying techniques can also be applied. For open-weight models, for example, developers can embed unique watermark patterns in the model weights themselves. Researchers are also developing new methods to infer the provenance of open-weight models that lack watermarks – for example, determining whether one model was created by fine-tuning another.

While model-tracking techniques help establish accountability and monitor misuse, they also raise potential privacy concerns. Overly broad monitoring could enable surveillance of legitimate research or user activity, creating tensions between accountability goals and individual or institutional autonomy.

Improvements in AI content detection techniques

Watermarks, metadata, and other AI content detectors help researchers track the spread of AI-generated content, study its impacts, and identify sources when harms occur.

Work on these techniques has advanced in three main areas since the publication of the last Report. First, there have been improvements in both cryptographic provenance methods, which use digital signatures, hashes, or encryption to verify authorship, and digital ‘watermarking’ methods that add subtle, distinctive patterns to AI-generated content with information about its origin (see Figure 4). These watermarks include distinctive word choices for text outputs, subtle patterns in pixels for AI-generated images and videos, and patterns embedded in audio waves for audio outputs. Recent studies have strengthened the reliability of these approaches. For example, one study combines cryptographic fingerprinting with semantic-aware encoding to make image provenance verifiable even after compression or some forms of editing. In another study, researchers improved text watermarking by embedding statistical signals across multiple vocabulary channels while preserving fluency.

Second, standardised file formats for AI-generated content that include information about how it was generated have improved.

Finally, researchers have continued to develop AI content detectors—systems trained specifically to distinguish AI-generated from human-created content based on statistical patterns.

Figure 4: Watermarking is one of several key techniques for studying the downstream uses and impacts of AI-generated content. Source: William Warby 2024.

Each of these techniques has useful applications but remains vulnerable to deliberate tampering. Recent studies demonstrate that watermarking and provenance signals can be removed, forged, or degraded through relatively simple post-processing or adversarial manipulation, indicating that defensive methods currently lag behind adaptive attacks.

Identity and accountability mechanisms are emerging for AI agents

As AI models and systems increasingly act autonomously online – they can now make purchases, send emails, and execute code – establishing clear accountability is becoming more important. If an AI agent causes harm or violates policies, investigators need to determine which system was responsible and under whose authority it was acting.

Recent work has begun on developing technical frameworks to address these challenges. Proposed approaches include giving AI agents secure digital identities that can be verified and audited, creating unique identifiers for specific AI system instances (for example, distinguishing between different conversations with the same model), and developing systems that allow users to set explicit permissions for what AI agents can do on their behalf while maintaining auditable logs of agent activities. However, these frameworks remain in early stages of development, and it is too early to assess whether they will prove effective or practical at scale. Real-world deployment will need to address challenges such as implementation costs, integration with existing systems, and resistance to tampering or circumvention.