Vibeleaderboard
← All Intel
Intel / article

OpenAI releases 719 AI-generated math manuscripts with Lean proofs

Source
github.com
Author
OpenAITop Viber
Date

Context

OpenAI has published a large body of mathematics written by an unreleased internal model, and the main thing to understand is how much of it has been checked. Its repository holds 719 manuscripts grouped into 372 families, where a family collects a principal result with its companion arguments, consequences or alternative proofs, so 719 counts papers rather than separate problems. About 42% of the top-line results come with formal proofs in Lean, a language in which a proof is written precisely enough for software to check every step. The rest are conventional write-ups, and OpenAI says some of these unformalized results could have issues, which it will try to fix quickly while it adds formalizations.

OpenAI says the collection came out of its model evaluations. After performance on its existing math benchmarks saturated, leaving those tests unable to show further progress, the company expanded testing to open research problems, and over the course of that the model was posed about 4,000 problems. Each result took on average three hours of ChatGPT Pro thinking compute, and OpenAI grouped the output into families and kept those that met a significance bar. Two pieces, a zero-free region for the Riemann zeta function and a proof of the Hodge Conjecture for CM abelian varieties, were produced outside that fixed procedure, and one Riemann zeta write-up was edited by a person for readability. OpenAI also released abridged summaries of the model's reasoning for ten of the families, among them the irrationality exponent of π and the Mahler conjectures.

In its announcement, OpenAI said it consulted the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study on how to release the results. The manuscripts and reasoning PDFs were not readable in the material available here, so nothing in this explanation judges whether any proof is correct. The repository is set up to change: corrections are recorded as new versions with earlier ones kept accessible, and changes, including withdrawals, are logged on its history page, covered in a later Intel item. This release is far larger than the ten results OpenAI published in August.

Terms in this piece · Glossary
  • eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
More from OpenAI
Recommended reads
Comments

Checking sign-in…

Loading comments…