OpenAI embeds outside safety evaluators inside its own labs
OpenAI signed deals with four external evaluators for the first phase of embedded assessments.

· Last verified: 17 Nov 2026 (OpenAI)

Summary
- OpenAI signed deals with four external evaluators for the first phase of embedded assessments.
- Apollo Research, METR, Transluce and Goodfire get checkpoints, source code, training data and monitoring systems.
- Reviewers now work on OpenAI machines rather than testing pre-release copies from outside.
The latest
Four independent evaluation groups will work inside OpenAI with access to model checkpoints, source code, training and evaluation data, monitoring systems and conversations with responsible engineers, under agreements announced Monday, November 16, 2026. OpenAI called the arrangement an ambitious first step toward deeper access and disclosure rights, and said it is working toward full embedded assessments spanning multiple developers.
Details
- The four groups: Apollo Research, METR, Transluce and Goodfire signed on for phase one, according to OpenAI. Evaluators work on site on OpenAI hardware, a sharp departure from the standard practice of handing outside testers a pre-release model copy and limited documentation.
- Apollo's mandate: Apollo Research said the same day it is embedding six evaluators to examine whether models attempt to subvert their own alignment and control mechanisms, or those of successor models. Its questions: does training incentivize the behavior, can evaluations elicit it, does it surface during internal deployment, and is the reasoning detectable.
- Apollo's access: Per Apollo, the team receives training code and relevant datasets, reinforcement-learning rollouts with reasoning traces and available reward signals, and the ability to query intermediate checkpoints. Access is limited to "only the agreed claims and only for the specific models and time periods agreed for the assessment." Apollo did not say when the trial begins.
- METR's work: METR is assessing dangerous or unintended behavior inside the largest reinforcement-learning run OpenAI has conducted to date, plus a second assessment of how well OpenAI's own systems detect that behavior. METR separately published a three-week red-team exercise on OpenAI agent monitors in October 2026.
- Transluce and 2027: Transluce is examining the quality and coverage of post-training alignment evaluations, including how those tests might shape release decisions in 2027. A follow-on assessment with Transluce running roughly eight weeks is planned for early 2027 with broader model access.
- Goodfire's role: Goodfire is placing interpretability researchers inside the company to monitor training runs and investigate undesirable behavior in model internals, according to OpenAI.
- The limits: Staff conversations are off the record, though evaluators can request documented data from OpenAI and publicly state when the company refuses to document something. OpenAI can withhold information for technical, security or intellectual-property reasons, and pledged not to withhold anything merely because it is unfavorable.
- Who evaluators may ask: Apollo's reviewers speak only with authorized staff designated by OpenAI, and may propose names. If an employee outside that circle raises confidential in-scope information, Apollo halts contact and resumes through OpenAI.
- The money: Apollo Research and Goodfire are paid; METR and Transluce work without a fee, with the option of accepting API or compute credits. Payment is not tied to findings. Disputes go to the safety and security committee of the OpenAI Foundation board.
- Publication rights: Evaluators decide what to publish and when. OpenAI keeps the right to redact confidential information and to request a delay to address security or safety problems first. Apollo can publish independent conclusions, but OpenAI can determine which evidence is available to support them.
Between the lines
The publication terms carry the tension. Apollo said genuine independent evaluation requires all data, records and documentation relevant to its assessment, plus the ability to share findings with an outside audience even over a developer's objection. Yet OpenAI retains redaction rights, delay requests, control over which staff evaluators may question, and discretion over which evidence can back a published conclusion.
What's next
Watch the start date for the Apollo trial, which has not been set, and the eight-week Transluce assessment planned for early 2027 with wider access. Also watch whether evaluators invoke their right to publicly flag a refusal to document.
Source: OpenAI