OpenAI says new models lift COBOL migration accuracy to 97.9%

Summary

• OpenAI published two benchmarks measuring model performance on COBOL, Fortran and other low-resource programming languages.
• GPT-6 Astra averaged 87.4% on MultiLangBench against 26.8% for GPT-5.6 Sol, the company said.
• Banks and governments running legacy mainframe code gain a public, testable yardstick before committing to migration.

The latest

Models can now rewrite COBOL programs into modern languages with 97.9% accuracy at higher reasoning effort, OpenAI said in an October 19, 2026 post on low-resource programming languages. The company built two evaluations covering 20 languages and published the setup, test suite and per-language results on GitHub. It cautions that production migrations still need expert review.

Details

• Why these languages: OpenAI said languages such as COBOL, Fortran, Visual Basic and OCaml offer far less code and documentation for models to learn from than Python or JavaScript, yet run critical infrastructure. Feedback from developers and enterprise customers shaped the work, according to the company.
• The scale cited: Citing IBM, OpenAI said COBOL underpins more than 40% of online banking systems, 80% of in-person credit card transactions and 95% of ATM transactions. Fortran, it said, is used in weather forecasting and nuclear safety analysis.
• The first benchmark: MultiLangBench asks models to build a SQLite-compatible database engine from scratch in a specified language. Tests are adapted from the SQLite test suite, and a reference SQLite build verifies the resulting database files. Coverage spans 20 languages including C, Go, Java, Lean, PowerShell, Rust, Scala, Swift and Zig.
• The headline numbers: At maximum reasoning effort, GPT-6 Astra averaged 87.4% success versus 26.8% for GPT-5.6 Sol, clearing 83% in 19 of 20 languages. COBOL was the weakest at 72%, OpenAI reported.
• Per-language gains: Against GPT-5.6 Sol, Astra moved COBOL from 10.9% to 72.1%, OCaml from 25.6% to 85.8%, Fortran from 21.4% to 83.5% and Visual Basic from 27.5% to 89.4%, according to the published results.
• The cheaper model: GPT-6 Luna averaged 40.3% at an estimated $0.77 per sample against $3.48 for GPT-5.6 Sol, roughly 78% less. Its per-language scores were far lower: COBOL 17.1%, OCaml 34.1%, Fortran 31.4%, Visual Basic 41.6%.
• The migration test: The second evaluation has models port COBOL programs to C++, Java, Kotlin, Python or Rust while preserving behavior. Hidden tests require byte-for-byte matching of observable output, and OpenAI said "a migration counts as successful only if all tests pass."
• Migration results: GPT-6.1 Sol reached 86.6% accuracy at low reasoning effort against 5.4% for GPT-5.6 Sol at the same setting, at 39% lower cost. At higher effort it hit 97.9%, which OpenAI describes as near saturation of the evaluation.
• What is available: The MultiLangBench setup, test suite and per-language results are on GitHub, and developers can test the latest models through the API on coding tasks. The post does not specify wider availability for GPT-6 Astra, GPT-6 Luna or GPT-6.1 Sol.
• The caveat: OpenAI said applying these migration capabilities to production software still requires expert review and testing against a system's business rules and dependencies.

Background

COBOL and Fortran date to the mainframe era and remain embedded in banking, payments and scientific computing. Institutions have long weighed replacing them against the risk that rewritten code behaves differently under real transaction loads.

Between the lines

The gap between Astra's 72% on COBOL in MultiLangBench and GPT-6.1 Sol's 97.9% on migration points to two different tasks: writing a database engine from scratch in an unfamiliar language is harder than translating existing logic into a well-documented one. The byte-for-byte pass requirement also means the migration score reflects exact behavioral matching, not approximate correctness.

What's next

Watch whether independent teams reproduce the MultiLangBench results from the published GitHub setup, and whether OpenAI announces broader availability for GPT-6 Astra, GPT-6 Luna and GPT-6.1 Sol beyond API coding tasks.

Source: OpenAI