← All IntelClip / OtherComputer use evolution: from single-action API to code execution
From Codex, Behind the Harness — Dominik Kundel, OpenAI · ≈8:12
“the recent models and the recent API shapes allow you to use code execution instead to actually do computer use, which means that the agent can script its own interactions with the uh with whatever computer implementation you want to have.”
“You can choose the language like JavaScript or Python and you are you have a much more flexible harness.”
“it actually interacts with a persistent node repl that gets persisted throughout different uh throughout the turns.”
What’s in it
- Explains how OpenAI's computer-use API moved beyond single-action calls
- Reveals Codex drives browsers via a persistent Node REPL
- Shows how code execution lets agents script their own Playwright actions
Clip transcript
Computer use is an interesting one because we actually introduced uh computer use in the responses API last year and it was fairly limited. It only allowed you to do one action at a time and um you had to basically declare that you wanted it to do computer use and from there you were up to actually implementing specifically the type of actions that were exposed to that tool. This was uh great in terms of like that state uh that point in the journey of of building agents, but since then we've evolved it and uh the recent models and the recent API shapes allow you to use code execution instead to actually do computer use, which means that the agent can script its own interactions with the uh with whatever computer implementation you want to have. You can choose the language like JavaScript or Python and you are you have a much more flexible harness. In fact, that's what we use for browser use. Um so, the what happens when uh Codex uses browser use is it actually interacts with a persistent node repl that gets persisted throughout different uh throughout the turns. And then it writes JavaScript um essentially Playwright code to interact with that uh browser instance in the node repl.
Comments
Checking sign-in…
Loading comments…