Money and business in the Middle East.

AI

OpenAI says Codex agents ran 1,700 ChatGPT performance investigations in 3 months

Coding agents are no longer just writing code at OpenAI — they are running the performance engineering loop end to end.

· Originally published by ontime+ · Last verified: 21 Nov 2026

Key Points

  1. OpenAI described an internal system where Codex agents diagnose and fix ChatGPT slowdowns automatically
  2. Agents root-caused over 600 of 1,700-plus investigations, roughly 17 automated investigations a day
  3. Engineers now mostly review and merge, with OpenAI studying whether agents could deploy low-risk fixes alone

The latest:

Coding agents are no longer just writing code at OpenAI — they are running the performance engineering loop end to end. In an engineering post published November 19, 2026, authors Okay Z and Cyrus Forbes described an internal Performance Factory built on Codex agents that detect ChatGPT regressions, identify root causes, propose fixes and verify recovery after deployment.

Details:

  • The numbers: According to the post, the Regression Workflow tracked more than 1,700 investigations in slightly over 3 months and pinpointed a root cause in more than 600 cases, averaging 17 automated investigations per day. Before Codex, the team had accumulated more than 30 separate performance tools.
  • The accuracy check: In a manual review of 100 agent-proposed root causes, OpenAI estimated 70% were accurate, while 30% needed refinement or did not fully establish causation. The post cites one case where worker startup time more than doubled, but gives no figures on latency, cost or time-to-fix.
  • How it evolved: OpenAI traced several stages: manual investigation before Codex; Codex as a partner under engineer direction; a shared perf-forensics skill storing the methodology; engineers acting as human relays pasting alerts into Codex and results into Slack; cloud-run automated investigations posting to Slack; then the Performance Factory itself.
  • The architecture: Four components, per the post: a task service with configurable workflows defined by config files and Markdown; a dispatch loop that picks up ready tasks and starts a Codex turn; an internal interface running Codex inside OpenAI’s network with access to code repositories, metrics, logs, traces, CPU profiles and memory snapshots; and event subscriptions.
  • The workflow: The pipeline runs detection by the system, root-causing by an agent, a proposed fix by an agent, review and deployment by an engineer, then recovery verification by an agent before closure. Agents operate under a service identity so their permissions are scoped and their actions distinguishable from humans.
  • Adversarial review: A separate Codex session tries to disprove each diagnosis. In one example, an agent tied ChatGPT conversation failures to image download errors; the reviewer noted an internal file-location service had already recovered while image requests kept failing, and reopened the investigation. Proposed fixes face the same adversarial pass.
  • After deployment: An agent checks whether performance returned to baseline. Full recovery closes the task; partial recovery closes the original and opens a new one for the residual slowdown; no improvement is logged and the regression returns to investigation. Each task generates a report with metric time series, flame graphs and traces.
  • Self-repair and evaluation: A daily agent reviews recent run logs for tooling and access failures, and when one error blocks multiple investigations it files a repair task for engineer review. Investigations are backtested against past production incidents with answers withheld, scored separately on onset timing, magnitude, affected requests and cause ranking.
  • What is next: The post lists proactive regression detection without alerts by comparing profiles across releases, a service rightsizing skill for over-provisioned autoscaling, an evaluation loop comparing agent fixes with what engineers merge, synthetic performance tests, and cross-workflow coordination. The system is internal, with no customer product or API mentioned.

Background:

OpenAI framed the shift as a response to its own development speed, arguing that the pace of agentic development has made traditional performance engineering approaches unsustainable. The Regression Workflow is the first pipeline built on the system.

Between the lines:

The authors locate today’s bottleneck in human review and merge, and say they are already studying whether agents can handle that themselves. Read against the 70% accuracy estimate and the adversarial review that reopened a wrong diagnosis, the engineer’s role is still the control point — the remaining human step is also the one OpenAI says it wants to test removing for low-risk fixes.

What’s next

Watch whether OpenAI publishes results from the planned evaluation loop comparing agent-proposed fixes with engineer-merged ones, whether proactive detection ships without alerts, and whether any low-risk fix is deployed without a human in the loop.

Read on ontime+ ↗

OpenAI says Codex agents ran 1,700 ChatGPT performance investigations in 3 months · INXEN