Hands-On With Gemini 4: Top-Tier Model, Weak Multi-Agent Harness
- Source
- x.com
- Date
I spent a decent amount of time with Gemini 4 before launch. Overall, Gemini 4 is a big jump over previous Gemini models and it is the first non-OpenAI/Anthropic model to pass the Fable/Mythos/Astra bar. It is also much more pleasant and more agentic then previous Gemini models, without the weird personality issues that you often found in Gemini 3 series models. In fact, a big advantage is that has a very different personality and sense of taste than Claude or Astra, making it a good addition to the AI world. On the usefulness side, in some areas it is at least at par with OpenAI and Anthropic (it is quite creative and a good writer), but in others it lags, including at complex multi-agent work. It has a tendency to be too certain about its conclusions and a little lazy, issues not helped by the Antigravity harness not being quite as good as the competition. Still, I expect to see rapid improvements in model abilities using Gemini 4 as a base. Here is a fun example: "create the ultimate brutalist city builder. you should have lots of controls over buildings in ways that are delightful and inuitive to people who just want to play, but are sophisticated and deep for people who…
Context
Ethan Mollick's early use of Gemini 4 produced a split assessment: he found it a large step up from earlier Gemini models and more agentic, and judged it at least on par with leading competitors for creativity and writing, while it lagged on complex work involving multiple agents. He also reported overconfidence and shortcomings in Antigravity, the environment in which he used the model. First-person assessment.
His account therefore supports separating writing ability from the reliability of a complete workflow. These are judgments from his prelaunch use; the post does not isolate how much of the weakness comes from the model versus its surrounding software. Reported experience.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- multi-agent — Using several AI agents on one problem — splitting work in parallel, checking each other, or filling different roles like planner and reviewer.
- eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
- agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
Checking sign-in…
Loading comments…