Parameter Golf with AutoResearch — Vayum Arora, Zhengyao Jiang, Dixing Xu & Dhruv Srikanth, Weco AI

Gives a practical recipe for agent-driven experiment search and names the failure points, especially reward hacking and unstable evaluations, that decide whether the loop produces real gains.

• Weco AI describes autoresearch as a loop where an agent proposes code changes, keeps those that improve a measurable score and discards the rest. Their Mini Parameter Golf setup makes training experiments small enough to inspect as they run.
• A useful setup needs a modifiable source file, a fixed evaluation command and a measurable goal. The presenters stress fast and stable evaluations, a clear metric direction, time limits, initial guidance and protection against reward hacking.
• Weco's experiment tree shows each attempt's code changes and score, letting users branch from promising work and steer a running search. A Flappy Bird exercise with a hidden validation set and edit limits illustrates designing evaluations that reward the intended behavior.

• eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
• AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.

• videoHow Secrets Leak Through AI Agents (and How to Stop It) — Venice AI
• videoFrom Zero to Leaderboard: Agent Evaluation — Wolfram Ravenwolf, Weights & Biases
• videoBuild a Platform and Watch It Burn — Michael Forrester, Accenture & Whitney Lee, Datadog
• videoContinuously Improving Agents with Langfuse — Annabell Schäfer & Lotte Verheyden, ClickHouse

• videoBeating RL With Reflection: GEPA and Optimize Anything — Lakshya A. Agrawal, GEPAAI Engineer
• videoBuilding Self-Learning Loops for Your Agent — Fuad Ali, Arize AIAI Engineer
• videoLet Your Agent Cook: Using Skills to Evaluate and Improve Your App — Ankur Duggal, Arize AIAI Engineer

Checking sign-in…

Loading comments…