Vibeleaderboard
← All Intel
Intel / article

Can AI help scientists identify promising drug candidates?

Source
openai.com
Date

Context

OpenAI has released MedChemBench, a test of whether AI models can do the reasoning medicinal chemists do when deciding which drug candidates to pursue. The hard part of that work is that improving one property of a molecule often worsens another: a compound may block a disease-related protein strongly but be poorly absorbed or broken down too quickly in the body. The 's 99 problems are drawn from 117 published papers and patents. Most give the model what researchers knew at a point in a real project, such as the target protein, their goals, the compounds made so far and selected results, then ask it to reconstruct molecular structures from drawings, explain results, rank or pick compounds, or predict measured properties. In one problem the model must predict, for three related compounds, how strongly each blocks a cancer-related protein called PRMT5, how easily it crosses a cell barrier and how quickly liver enzymes break it down. The later experiments, withheld from the model, showed a trade-off: one compound blocked PRMT5 at a lower concentration, while another crossed the barrier more readily and broke down more slowly.

At the reasoning setting OpenAI calls xhigh, the newly released GPT-6.2 Astra earned 84.5% of the available credit, against 83.3% for GPT-6 Astra, 82.0% for GPT-6.1 Sol and 58.2% for the older GPT-5.6 Sol. The overall score hides an uneven profile. GPT-6 Astra earned 95.4% of the credit for reconstructing molecular structures but only about half for selecting compounds and predicting numerical properties, the parts closest to the decisions chemists actually make. Getting the structure right comes first: on one problem that required preserving an 18-atom ring, GPT-6 Astra was correct in all ten attempts, while GPT-5.6 Sol failed all ten, in one case producing a 15-atom ring, which is a different molecule.

OpenAI states the limits itself. The results do not show whether models can discover drugs or improve laboratory decisions, the models may have seen the published sources during training, and each answer was a single attempt without tools, feedback or new experiments. All the reported scores are OpenAI's measurements of its own models. What the benchmark shows is where current models fall short, in choosing compounds and predicting their properties, rather than that they are ready to guide a drug-discovery program.

Key takeaways · AI-distilled
  • MedChemBench has 99 problems drawn from 117 papers and patents with 761 distinct molecular structures. Models get the available at a point in a research program and are graded on structures, reasoning, compound selection and property predictions.
  • At the xhigh reasoning setting, OpenAI reports GPT-6.2 Astra earned 84.5% of available credit, GPT-6 Astra 83.3%, GPT-6.1 Sol 82.0% and GPT-5.6 Sol 58.2%. Higher reasoning settings raised scores, but large gaps between models remained.
  • GPT-6 Astra earned 95.4% of the structure-reconstruction credit at xhigh but only about half the credit for selecting compounds and predicting numerical properties.
  • On a problem requiring an 18-atom ring to be preserved, GPT-6 Astra rebuilt the molecule correctly in all ten responses while GPT-5.6 Sol and Luna failed in all ten; illustrated answers showed a 15-atom ring and a molecule split into two fragments.
  • OpenAI says the results do not show that models can discover drugs: prior exposure to the published sources cannot be ruled out, and each response was a single attempt without tools, feedback or new experiments.
Terms in this piece · Glossary
  • benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
  • context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Recommended reads
Comments

Checking sign-in…

Loading comments…