Vibeleaderboard

Ivan Leo

Viber rank800

Ivan Leo works in Developer Experience at Google DeepMind, helping developers build with Gemini and evaluate autonomous agents. His earlier work includes Manus’s action engines and the Instructor library for structured language-model outputs.[1]

Start with Write Stupid Evals →
Ivan Leo

Career

Leo began in frontend engineering at Credit Suisse in Singapore and learned machine learning through independent study and a paper-reading group. He later worked on Instructor with Jason Liu. He joined Manus on July 27, 2025, building Mail Manus, file suggestions, API integrations and payments support before moving to Google DeepMind.[2][5][1]

Contributions

Make an evaluation small enough to run often

In Write Stupid Evals, Leo starts with checks a developer can inspect: valid JSON, required fields, retrieval hits and correct tool selection. His text-to-SQL example measures whether the required snippet appears among the retrieved candidates, allowing changes in retrieval quality to be weighed against latency. He then extends single-tool checks to multiple tools and argument validation while retaining the earlier baseline. The aim is a fast local feedback loop that grows alongside the application.[4]

Represent an agent session as an interaction

In his Google DeepMind talk, Leo presents the Interactions API’s server-side state and interaction identifiers, typed multimodal outputs and combined built-in and custom tools. The publisher’s chaptered description also covers Managed Agents: persistent remote sandboxes, environment identifiers, loadable sources and a proxy for injecting credentials. These interfaces organize state and execution around a continuing agent task.[6]

Ideas

His Manus retrospective treats delivery as work extending beyond a successful deployment. A working email-triggered prototype exposes product questions early; watching colleagues use it reveals confusing interfaces. He describes learning Go while repairing the API’s attachment and connector behavior, and using support-ticket clustering to understand failure modes across languages. These examples connect implementation, documentation, integration testing and user feedback.[5]

Tools, writing & talks

Writing, research & talks

  • Start here

    An authored tutorial moving from simple assertions to retrieval and agent-tool evaluations.

    Take one unreliable behavior in an existing application and express it as an assertion or a small labeled test set. The Python examples assume familiarity with model calls; keep that baseline when adding more complicated agent checks.

    Author[4]
  • A talk introducing session state and managed execution in the Gemini agent interfaces.

  • A concrete retrospective on prototypes, API integration, support analysis and ownership after launch.

    Author and participating engineer[5]
References

Latest source check 2026-10-30

  1. [1] Owned current biography

    ivanleo.com · Checked 2026-10-30

  2. [2] Original career and speaker biography

    ai.engineer · Checked 2026-10-30

  3. [3] Owned professional account

    x.com · Checked 2026-10-30

  4. [4] Original authored evaluation essay

    ivanleo.com · Checked 2026-10-30

  5. [5] Original authored Manus engineering retrospective

    ivanleo.com · Checked 2026-10-30

  6. [6] Original Interactions API talk and publisher description

    www.youtube.com · Checked 2026-10-30

Ivan Leo | VibeLeaderboard