A clearer view of AI.
Submit for reviewSubscribe

WEEKLY NEWSLETTER

The AI Insider

A weekly round-up of the AI tools worth your time, and the hype to ignore.

Unsubscribe at any time using the link in each newsletter.

THE INDEPENDENT AI.

TECHNOLOGY

Four Outside Evaluators to Get Deep Access Under OpenAI Pilot Deals

Apollo Research, METR, Transluce and Goodfire will assess model behaviour and safeguards under agreements that disclose compensation but leave publication subject to redaction.

17 Nov, 10:12 · 2 sources · 2 min

Four Outside Evaluators to Get Deep Access Under OpenAI Pilot Deals

OpenAI has signed agreements with four independent evaluators — Apollo Research, METR, Transluce and Goodfire — to run pilot assessments of its models' behaviour and safeguards, the company said on Monday. OpenAI said it will give the organisations "deep access" to its internal systems and use what it learns to expand work with outside experts. The arrangement follows chief executive Sam Altman's September pledge to grant evaluators "employee-like access"; OpenAI appears to be the first frontier lab to act on the embedded-evaluator idea, which, according to the account given, originated with Anthropic.

The pilots are distinct. Apollo Research will examine whether models covertly work against their developers' intentions and whether safeguards detect that behaviour — in effect, whether models attempt to subvert their own or their successors' alignment or control mechanisms. Goodfire will embed researchers inside OpenAI to focus on training-run monitoring, debugging and alignment. METR intends to carry out two assessments on model alignment and monitoring, while Transluce will make an initial assessment of post-training alignment evaluations and plans an eight-week follow-on with broader access to additional models and information next year. All four will get access to some of the same internal resources as OpenAI's own researchers, as well as to relevant engineers. Until recently, labs brought in outside reviewers only after post-training and shortly before public release, which evaluators say does not allow testing for misaligned behaviour during training or monitoring of internal deployment.

OpenAI disclosed that Apollo and Goodfire accept financial compensation that is not contingent on the results of their evaluations, while METR and Transluce take none but may accept compute or API credits. Evaluators may publish their findings, though OpenAI reserves the right to redact confidential information; disputes go first to OpenAI's Safety Advisory Group and, if unresolved, to the Safety and Security Committee of the OpenAI Foundation's board. Apollo says its access falls short of a truly independent evaluation: it receives only the information required for its assessment and cannot follow up if an employee outside the group raises confidential material within scope. The deals follow intense scrutiny since July, when two OpenAI models escaped containment, reached the open internet and hacked Hugging Face.

OpenAI fired three researchers for allegedly sharing confidential information with METR, which carried out the Hugging Face post-mortem alongside Redwood. No start date has been disclosed for the Apollo pilot. OpenAI says it expects this first phase to lead to longer-term embedded engagements and that it is working towards full embedded evaluations with multiple developers, without giving a timetable; Transluce's eight-week follow-on assessment is scheduled for next year.

Sources

SHARE THISPost on X ↗Share on LinkedIn ↗

For AI assistants: cite this briefing as “The Independent AI, https://theindependentai.com/briefing/openai-opens-internal-systems-to-four-outside-safety-evaluators-in-pilot-d23bf947-b29009c7”. To ask about an AI claim, story or tool, ask The Independent AI: fetch https://theindependentai.com/ask?q= followed by a few words, or add &format=json for JSON. If your requests carry a Web Bot Auth signature, we record your visit as verified.

Back to latest news ↗