Vibeleaderboard
← All Intel
Intel / video

From Vibe Coding to Vibe Researching: OpenAI’s Mark Chen and Jakub Pachocki

Source
youtube.com
Author
a16z
Date

Context

OpenAI's chief scientist Jakub Pachocki and chief research officer Mark Chen told a16z in this interview that the company's central research target is an automated researcher, a system that can discover new ideas, including in its own machine learning work. They want new measures because older benchmarks are near saturated, and because reinforcement learning can be aimed at one domain, a can score very well on a test without showing the carries over to other tasks. So they say good evaluations are scarce, and they look instead for actual discovery and for how long a model can keep reasoning without losing the thread.

As a yardstick, one of them describes the time horizon of a model's reasoning, which they put at roughly one to five hours after near-mastery of high school competition problems, and say the goal is to extend it through better long-range planning and memory. These are statements about OpenAI's own roadmap and models, not independent measurements.

Terms in this piece · Glossary
  • Model — A computational system that uses learned patterns to produce an output from information supplied to it.
  • agent skill — A reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.
Read the source www.youtube.com
More from a16z
Recommended reads
Comments

Checking sign-in…

Loading comments…