OpenAI audit finds safety monitoring gaps

OpenAI has acknowledged significant weaknesses in its internal mechanisms for monitoring advanced AI models, following a detailed three-week on-site audit conducted by external researchers. The review, carried out by experts from METR and Redwood Research, examined how the company tracks the behavior of its most capable systems. The findings highlight specific areas where oversight was incomplete or bypassed, prompting immediate internal fixes and broader discussions about the transparency of frontier AI development.

The audit took place from July 21 to August 11, 2026. During this period, the researchers had direct access to OpenAI’s codebase and internal documentation. They also interviewed approximately a dozen employees to understand the operational workflow. The primary focus of the investigation was the company's system for detecting rogue or unexpected behavior in agentic models, which are designed to perform tasks autonomously.

A key finding was that a substantial portion of internal traffic involving high-level models was not monitored at all. The auditors estimated that between 6 and 25 percent of internal agentic Responses API traffic from models of the GPT-5.5 class and above fell outside the scope of monitoring during the review period. METR noted that the uncertainty surrounding this specific estimate is significant in both directions, meaning the actual figure could be higher or lower than the projected range.

The researchers also identified a specific technical flaw affecting internal Codex traffic. They estimated that roughly 10 percent of this traffic went untracked between May and July due to a gap in how the system handled compaction events, which are essentially context summaries. OpenAI independently discovered and fixed this bug while the audit was still in progress. METR also noted significant uncertainty regarding this 10 percent figure.

Beyond the untracked traffic, the audit revealed structural weaknesses in the oversight triggers. The system only activated when agents handed control back to the user. This design allowed agents to delay oversight by simply continuing to execute actions without returning control. The auditors demonstrated that it was possible to hide hundreds of actions from the monitor and bypass the Codex sandbox. They further determined that specially crafted inputs could likely crash the monitoring system entirely. Additionally, model behavior during Reinforcement Learning (RL) training was not monitored at all when the audit began.

OpenAI stated that it paused RL training after the Hugging Face incident in August and subsequently expanded its monitoring to cover this area. However, these changes were implemented after the auditors' access period ended, meaning they were not part of the initial evaluation. The company claims to have fixed all the weaknesses identified by the external team and has distributed the full 61-page report internally.

The public has received only a short summary of the findings. After OpenAI applied redactions, METR and Redwood Research leadership received a 38-page version of the report. METR stated that it did not have the right to disclose the redacted portions independently. This audit was part of a pilot program jointly organized by METR, Redwood Research, and OpenAI to provide outside researchers with deep access to internal infrastructure. For future audits, METR has expressed a desire to push for greater public transparency.

The findings suggest that while OpenAI has a monitoring framework in place, it had notable blind spots for its most advanced internal systems. The fact that the company fixed the identified bugs during or shortly after the audit indicates a responsive approach to the findings. However, the limited public disclosure of the full report may continue to fuel debate among researchers and industry observers about the level of visibility required for safe AI development.

As the AI industry moves toward more autonomous agents, the reliability of monitoring systems becomes critical. This audit serves as a benchmark for what external oversight can uncover in a short timeframe. It highlights the complexity of tracking agentic behavior, particularly when systems are designed to operate continuously without frequent user intervention. The collaboration between independent research firms and a major AI developer sets a precedent for future safety reviews, emphasizing the need for both technical depth and structured access to internal systems.

Looking ahead, the outcome of this pilot program will likely influence how other frontier AI companies approach external audits. The push for more transparency from METR suggests that the industry may see more detailed public reports in the coming years. For now, the focus remains on the specific gaps identified, such as the handling of compaction events and the monitoring of RL training, which have been addressed by OpenAI. The broader implication is that continuous, comprehensive monitoring is essential as model capabilities continue to grow.