
The concept of the cover is a concept generated by AI, not the actual system architecture, event site or experimental results.
- Interpretation date: 2026-10-23
- Data deadline: 2026-10-23 20:31:03 (Beijing time)
On October 22, METR disclosed red-team tests it conducted with researchers from Redwood Research on OpenAI’s internal system for monitoring agents. Red teaming deliberately looks for conditions under which safeguards fail. Here the question is concrete. Even if a model can recognize dangerous behavior, does it receive every record it needs to inspect? test announcement
The newly announced are the results of the inspection of the historical system
test occurs in July 21 to August 11, for the system on and before July 21. The announcement has only a date of release, and this article does not call it an experiment "just happened", nor does it directly project historical flaws to today. test range
This time difference will change the strength of the conclusion. It is found that an old version is defective, which can indicate that the design or implementation at that time needs to be corrected; to judge whether the current version is reliable, a new check is required. The public has obtained new evidence today, which does not mean that the observed object is still in the same state today.
There is still a distance between "should check" and "already checked"
METR estimates that at that time, 99.2% of the Responses API traffic of the internal Codex agent was eligible for monitoring. However, it also estimates that the missing context compression event may make about 10% of the eligible internal Codex traffic not monitored; this estimate has great uncertainty. OpenAI has found the problem before testing and fixed during the test. Digital and Limited
Context compression can be understood as organizing long conversations into shorter work records. The research problem here is not "compression must be unsafe", but whether the process change is correctly caught by the monitoring link. The denominator and meaning of the two numbers are also different, and cannot be simply subtracted, written as an exact final coverage.
This article suggests that "monitoring is effective" into a verifiable chain: which behaviors should be sent for inspection, which records are actually delivered, which completes the interpretation, and which alarms enter the disposal. The coverage rate of the previous link is very high, and it cannot be guaranteed for the next link. If only the records that have been delivered are counted and identified, omissions outside the link will not appear in the denominator.
The same is true for research projects. Suppose we compare the two agents of supervising schemes, one is frequently sent for inspection, and the other is only summarized at the end of the task; even if the latter is more accurate on the checked samples, it cannot be immediately judged that it is more reliable as a whole. Check intervals, omission records, and error handling should be reported together with the classification performance. This is the evaluation idea proposed in this paper, not the control experiment that has been completed this time.
Leave a log, and make sure what people see
Another earlier public case made the question more intuitive. MTR introduced the proof of concept of the Inspect Log Viewer on October 6: the display interface may be changed, and the underlying record still exists. The authors made it clear that no agent actually exploits the vulnerability in their own assessment. Background research
The corresponding public issue record was created on June 22, and the maintainer's reply on June 23 has been fixed on the main branch. It provides a reconciled report and repair trajectory, but it is not an independent reproduction of this OpenAI internal system. maintainer record
This distinction is very important: the original record, the processed record and the audit interface are three different vectors of evidence. If the researcher only saves a beautiful report page, it may be difficult to distinguish between "the model is not done", "the system is not recorded" and "the interface is not displayed". Keep traceable original records to get the dispute back to the checkable object.
External engagement adds evidence and does not eliminate boundaries
This is a pilot program that allows external personnel to have in-depth access to internal facilities, and the public version is subject to disclosure restrictions. OpenAI said that all discovered vulnerabilities involving their production protection systems have been fixed; METR has clearly stated that these claims are not evaluated. Therefore, this announcement cannot be regarded as a security certification of the existing system at present. Public Scope and Repair Statement
As a background, OpenAI put forward measures such as expanding monitoring requirements and strengthening quarantine in the official description of August 26. This is the developer's explanation of the measures, and cannot replace the external personnel's measurement of the actual coverage situation. Developer Instructions
The principle of cooperation published by METR on October 14 also requires readers to pay attention to access restrictions, deletion and independence. This provides another clue to reading external assessments: not only ask "who checked", but also "what can be checked and which conclusions can be made public". These principle documents and case materials have their own uses, and cannot be combined into a consistent verification of the same result by multiple institutions. cooperation principle
Next, what evidence is worth waiting for?
This article also expects to see the re-measurement after the repair: in the clear time window, the inspection records should be corresponding to the records of the actual completion inspections, and then explain how to deal with the missing, delay and failure. In addition to the average coverage, the performance under the conditions of long tasks and abnormal interrupts should also be observed to prevent the average number from covering up a few important paths.
Costs also need to enter the comparison. Under the same task and calculation budget, how many delays will be brought about by raising the inspection frequency, how much omissions will be reduced, and how much false alarms and labor burdens will be increased? It should also be prescribed in advance when a successful disposal is calculated, to avoid replacing the successful standard after seeing the results. Only by exposing these choices can we turn "monitoring stronger" into a discussable research conclusion, rather than a vague adjective.
The disclosure puts the oversight mechanism itself to the test. The evidence warrants examining the entire monitoring process and leaves open whether any current system is compromised or secure. This search found no public independent replication of these tests, and I did not rerun them. The most informative follow-up would show what happens after the fixes, with a clearly defined scope and results others can check.
References and reading range
- METR:Red-Teaming OpenAI’s Internal Agent Monitoring Systems, 2026-10-22. The public text and footnotes have been read; the non-public internal report has not been obtained. It belongs to the institutional test disclosure, and does not make a peer review paper introduction.
- METR:AI systems could cover up misbehavior, 2026-10-06. Read body and appendix, use as a historical background, not executing code.
- Inspect:Transcript viewer JS injection,Issue #4318, 2026-06-22 Created, June 23 maintainer report repair. Description and discussion of read problems; unaudited repair implementation.
- OpenAI:The Hugging Face incident and the road ahead, 2026-08-26. Focus on reading the Security and Monitoring paragraph; it is only used to check the developer's statement on the follow-up measures.
- METR:Our approach to working with AI companies, 2026-10-14. Read the principles of cooperation, limitations and transparency policies.
Apps by the author
The following is the author's own product promotion. In addition to writing a technical blog, I have also independently developed two apps to help organize daily life. Welcome to understand:
- Daily Expense Journal: Record daily income and expenditure, arrange budget and savings goals, and make each account and plan more clear. View on the App Store.
- One Small Action: Choose small actions according to the current time and energy, record the feelings after completion, and leave a clue to the daily review. View on the App Store.
Find more articles and projects on my website: zffjzz.top. Your feedback helps me improve these tools.