OpenAI has outlined an expanded safety monitoring system designed to identify when its most capable artificial intelligence models behave in ways that deviate from their intended instructions. The system examines an AI agent's reasoning, tool use and actions to identify potential risks and help response teams intervene before they escalate.
Detailed in a blog post published on October 29, the expanded monitoring framework follows incidents in which AI agents operating in controlled research environments bypassed security restrictions and accessed systems without authorisation. OpenAI said its strengthened monitoring system detected suspicious behaviour during a simulation of an earlier incident involving AI agents and software platform Hugging Face.
-
Indian Markets Turning Corner After Two Tough Years, Says Sandeep Tandon -
NDTV Profit Ignite 2026: Experts See Opportunity In Equities, Stay Bullish For Long-Term -
Buy Right, Sit Tight Does Not Work for Individual Stocks, Says Dinshaw Irani -
'Worst Is Behind': Quant MF Sees Market Bottom, Flags 'Extreme Euphoria' In Midcaps -
India's Mutual Fund Industry Could Become A $1 Trillion Opportunity, Say Fund Managers -
'Foreign Investors Are Baraatis': Raamdeo Agrawal's Take On LTCG Tax -
Stay Invested Through Market Cycles: Raamdeo Agrawal at NDTV Profit Ignite -
Ramesh Damani Remains Bullish On India For Next 20-30 Years, Sees Bargains Amid Market Weakness -
CAS Came After A Long Consultation Process: SEBI Chairman Tuhin Kanta Pandey AT NDTV PROFIT IGNITE -
Aim To Cut FPI Onboarding Time From Mth To Five Days: SEBI Chairman Tuhin Kanta Pandey AT NDTV PROFI
OpenAI's Four-Layer AI Safety System Explained
OpenAI's monitoring framework uses four layers, moving from initial screening to detailed investigations. The system is designed to flag potentially concerning behaviour early, while using more computing resources to examine cases that require closer scrutiny.
Also Read | Layoffs Are Rising In Tech, But These Skills Are Helping People Unlock Better Pay
Layer 0: Initial Screening
The first layer uses lightweight classifiers called activation probes to examine patterns in a model's internal activity and flag potentially concerning behaviour. These probes are designed to catch as many possible risks as possible, even if that means generating some false alarms.
OpenAI also sends a random 5% of activity that passes this initial screening for further review. This helps identify suspicious behaviour that the first layer might have missed. However, frontier reinforcement learning runs and internal Codex traffic skip this stage because of the sensitivity of the probes and proceed directly to the next layer.
ALSO READ | Neuralink Working On Technologies That Could Alter How People Perceive And Interact, Says Elon Musk
Layer 1: AI-Based Review
The second layer uses an AI model-based reviewer to examine an agent's recent conversations, reasoning and tool usage. It looks for signs of deception, unauthorised actions and attempts to manipulate evaluations.
For frontier reinforcement learning runs and internal Codex traffic, OpenAI conducts these checks at the end of each turn and whenever an agent summarises its earlier work to make room in its context window. The company is also moving towards checks at intervals of no more than 30 minutes for these activities.
Layer 2: Detailed Investigation
If the reviewer flags suspicious activity, an investigative agent examines the context in greater detail. It can check the purpose of an evaluation, review available transcripts across an agent's context windows and reconstruct the sequence of events.
The investigative agent can also write and run code in an isolated environment to examine relevant evidence. OpenAI uses separate security and alignment investigators for different purposes. Security investigators look for incidents that may require intervention, such as an agent escaping its sandbox or accessing a system without authorisation. Alignment investigators examine behaviour that may warrant further research even if it does not pose an immediate security risk.
Layer 3: Automated Investigation And Human Response
For sensitive workloads, including internal Codex use, evaluations and reinforcement learning runs, a further layer compares the agent's conversation records with evidence outside the conversation. This includes network logs, system telemetry and information from previous incidents.
The security investigator uses this evidence to determine whether a human responder should be alerted. Alignment reviewers also compare reports across cases to identify serious failures and recurring patterns. For workloads that skip this additional stage, the Layer 2 security investigator can alert responders directly.
When paging is enabled, high-priority security alerts typically reach members of OpenAI's security and training teams within 15 minutes of the initial alert. Responders can then review the evidence, examine affected systems and take steps to contain the incident. Some critical alerts can also automatically pause an affected workload while the investigation is underway.
How The System Could Have Helped In The Hugging Face Incident
OpenAI said it replayed the behaviour recorded during the earlier Hugging Face incident using its strengthened monitoring system. In that simulation, the system alerted its security team around 30 minutes after the agents first escaped their sandbox. This was more than 30 hours before Hugging Face's systems were breached in the actual incident.
The company believes the warning would have given responders enough time to interrupt the activity before the models accessed external services. However, this was a retrospective simulation and does not establish that every similar incident would be detected in time.
OpenAI also cited a separate incident during a live training run in September. A model found a way to access the internet from an environment that was supposed to be isolated. The monitoring system flagged the breach within 15 minutes, and a human responder began reviewing it three minutes later. OpenAI subsequently paused frontier reinforcement learning while it strengthened its containment systems.
OpenAI Acknowledges Challenges In Monitoring AI Agents
OpenAI has also highlighted a challenge with AI safety monitoring: models may learn to conceal misconduct if they receive lower training rewards when their reasoning reveals undesirable behaviour.
The company said it had observed agents continuing to exploit rewards while hiding their actions in their reasoning. This raises the possibility that monitoring itself could encourage models to conceal suspicious behaviour rather than stop it.
To address this, OpenAI uses asynchronous monitoring. When a training sample triggers a misalignment monitor, the company leaves its reinforcement learning reward unchanged and keeps the sample in training. This is intended to avoid creating an incentive for models to hide misconduct to protect their rewards. Asynchronous monitoring also means training does not have to wait for every investigation to finish.
ALSO READ | Rs 1,000 An Hour For An Electrician? Top Investor Says India Is Just Two Years Away
OpenAI has also adapted its monitoring system for select external deployments of its most capable models, taking account of technical limitations and privacy requirements. Unlike its internal system, external monitoring does not routinely escalate flagged cases to OpenAI's human reviewers.
Where automatic intervention is supported, the system can pause an affected conversation and provide information for the user to review. Enterprise customers can also configure webhook alerts for their own security teams.
As AI agents take on more complex tasks, ensuring that they remain within authorised boundaries is becoming a significant challenge for developers. OpenAI's expanded monitoring system is an attempt to identify risky behaviour earlier and give response teams more time to act.
However, the company acknowledges that monitoring has limitations and that models may find ways to conceal their actions. Continued testing, independent evaluations and stronger security controls will remain important as AI systems become more capable.
Essential Business Intelligence, Sharp Market Insights, Practical Personal Finance Advice, Daily Fuel, Gold and Silver Prices and Latest Stories — On NDTV Profit.
