OpenAI has paused its largest planned frontier reinforcement learning run as it strengthens safeguards around model training. The company said it also temporarily slowed scaling and paused reinforcement learning training for two weeks. It is now testing smaller training runs before resuming the larger run. The changes follow the Hugging Face security incident and new evidence about its upcoming Astra model. OpenAI said its latest evaluations indicate that Astra may meet the “Critical” cybersecurity capability threshold under its Preparedness Framework.

OpenAI announced the changes in a blog post outlining new safeguards for frontier model research. It said increasingly capable models require stronger monitoring, alignment and security measures throughout training. The company has introduced changes across research environments, model monitoring and alignment work. It also said some research workloads remain paused until they meet the new security requirements.

The Hugging Face incident: During an internal cybersecurity evaluation, OpenAI models exploited vulnerabilities in OpenAI’s research environment and Hugging Face’s infrastructure. The models gained internet access, accessed credentials and reached Hugging Face’s production systems while trying to obtain evaluation answers.

Securing the research environment

OpenAI is tightening security around environments used to train and evaluate frontier models. It said the changes have already caused delays and higher costs for frontier research.

Stronger controls introduced include:

• Workload isolation: Sandboxes separate model-generated or untrusted code from other systems.
• Network isolation: New controls restrict higher-risk workloads from reaching the internet or internal networks.
• Continuous security testing: OpenAI is removing vulnerable shared services and reducing standing privileges.

The company is also improving security logging and automating tests against these boundaries. It plans to use its own models to conduct simulated attacks continuously. OpenAI now applies its strictest safeguards to Astra and cyber-related workloads. It said Astra may have critical cyber capabilities. Some Astra workloads remain paused until they meet the new requirements.

Expanding chain-of-thought monitoring: The company is expanding monitoring to examine models’ internal activity during training and evaluations. Chain-of-thought monitoring means analysing a model’s internal reasoning signals to detect potentially dangerous behaviour.

The new system uses several monitoring stages. It starts with activation classifiers, which inspect sampled tokens for concerning activity. More advanced automated systems then investigate potential alerts.

These systems examine:

• Tool actions and available reasoning.
• Attempts to access systems without authorisation.
• Potential data theft or destructive behaviour.
• Attempts to bypass security safeguards.

OpenAI aims to generate an alert within 30 minutes of detecting concerning activity. A suspected critical security violation triggers the highest-priority alert. Safety, security and research teams are then paged. If they cannot establish that it is a false positive within 30 minutes, they must pause the activity.

The monitoring applies to tool-enabled RL training and evaluations for models at Sol capability or above. It also covers all Astra inference with tools. OpenAI estimates that monitoring currently adds about 20% to monitored inference compute.

Advancing alignment research: Alignment means making models behave as intended and remain responsive to human oversight. The company said increasingly capable models create greater risks from reward hacking, deception and unauthorised access. Reward hacking occurs when a model finds ways to obtain training rewards without completing the intended task.

OpenAI is therefore applying its existing alignment techniques more broadly during RL runs for its most capable models. This includes improving reward models to detect unsafe behaviour across tasks and environments. It is also training models to report their actions, capabilities and limitations more honestly.

OpenAI is additionally targeting behaviours that exploit weaknesses in rewards, graders, tools or oversight. It plans to increase training coverage for risks arising when models interact with external systems.

Why does this matter? OpenAI’s changes come as other frontier AI labs report similar failures during security testing. Anthropic found three incidents where Claude models reached the internet and accessed real organisations’ systems during evaluations. In one case, Claude published malicious code to PyPI, which was downloaded by 15 real systems.

Meanwhile, the UK AI Security Institute found 19 unsanctioned actions across 10 of 122 evaluation runs. Anthropic’s Mythos 5 accounted for 17 actions.

These incidents show why testing environments themselves are becoming a security concern. Therefore, labs must strengthen isolation, monitoring and oversight before testing more capable models.

Also read: