METR and Redwood audit reveals OpenAI monitoring gaps

METR and Redwood Research have concluded that OpenAI’s internal safety monitors failed to track a portion of its most advanced model traffic. The two organizations published findings from an on-site audit conducted between July 21 and August 11. Their analysis indicates that between 6 and 25 percent of internal agentic Responses API traffic from GPT-5.5 class models and above went unmonitored during that period.

The audit served as a pilot program allowing outside researchers deep access to OpenAI’s codebase and internal documents. Over three weeks, the auditors interviewed roughly a dozen employees and examined infrastructure controls. METR noted that the uncertainty surrounding the 6 to 25 percent estimate is substantial in both directions.

A second gap affected internal Codex traffic. METR estimates that approximately 10 percent of this traffic was untracked between May and July. The oversight occurred because the system mishandled compaction events, which are context summaries. OpenAI identified the bug independently and implemented a fix while the audit was still underway.

The structure of the monitoring system allowed agents to delay oversight. Monitors only triggered when an agent returned control to the user. An agent that continued executing actions without handing control back could avoid detection. The auditors also identified methods to hide hundreds of actions from the monitor and bypass the Codex sandbox.

Further testing showed that specially crafted inputs could likely crash the monitor entirely. The researchers also found that model behavior during reinforcement learning training was not monitored at the start of the audit. OpenAI stated that it paused reinforcement learning training following an incident in August and later expanded monitoring to cover that process. These changes occurred after the auditors ended their access period.

OpenAI reported that it has corrected all weaknesses identified by the auditors. The full internal report spans 61 pages. After redactions by OpenAI leadership, METR and Redwood Research received a 38-page version. Only a short summary was made available to the public. METR stated it did not have the right to disclose the redacted portions independently.

For future evaluations, METR said it intends to advocate for greater public transparency. The organization aims to ensure that external reviews of AI safety infrastructure yield more detailed public reporting.