Introducing Gemini 4 Hydrogen

Context

Most of the safety claims in Google DeepMind's launch post for Gemini 4 Hydrogen cannot yet be checked against a published method. The post is the same announcement Google ran on its main blog, where Hydrogen's price and benchmark claims are explained. Its safety section compares Hydrogen with Gemini 3.8 Flash, Google's previous fast model, on how it behaves as an , meaning when it calls tools and edits code with little supervision. Google reports up to 81% less "overreach" and up to 79% less "under-service", terms the post does not define, and 12 times fewer misaligned actions and task shortcuts in tool-use and coding evaluations. It also reports a 96.7% score on an internal honesty evaluation and more than 2.3 times better resistance on the Gray Swan benchmark for indirect , an attack in which instructions hidden in a web page or file the model reads try to take over its task.

The model card that the post points to for details contains none of those agent figures. Its safety results are a short table of automated evaluations, which the card says are not human evaluation or , and it defers the rest to a Safety & Security Report it says will be available "in the coming days". Until that report appears, these numbers are Google's own account. The post names the honesty test as internal and says nothing about how the overreach and misalignment tests were built, and it gives the Gray Swan result only as a ratio, without the underlying scores or test setup.

What the card does publish is mixed. Against Gemini 3.8 Flash, Hydrogen's adherence to Google's safety policies on prompts that combine images and text rose 7 percentage points, while text-only and multilingual results stayed similar within confidence intervals. Unjustified refusals, where the model declines benign or borderline requests it could safely fulfil, rose 5.57 points, particularly for prompts with images. That figure sits beside the post's claim that false refusals on HealthBench Pro stay under 0.4%, but the two measure different things: the post's number concerns telling real biosecurity risks apart from legitimate medical and scientific questions, so both can be true. The card also reports that Google's targeted tests found Hydrogen below the biological and cybersecurity capability levels its Frontier Safety Framework treats as critical, though it reached the lower alert thresholds for both. For anyone weighing whether to let Hydrogen act on tools and files, the published evidence so far shows better handling of image prompts and more over-refusals, while the improvements in agent behavior remain claims to check against the full report.

Key takeaways · AI-distilled

Google says Gemini 4 Hydrogen scores 51 on the Artificial Analysis Intelligence Index at its default high thinking level, edging the frontier Gemini 4 Argon at 50, and reaches 54 at the new xhigh level, priced flat at $1.50/$7.50 per million tokens across its 1M .
Antigravity's new Orchestrator switches between Carbon and Hydrogen inside one session. Google reports it reached near-Carbon task success on Terminal-Bench 4.0 while using 24% less quota and finishing 19% faster on average.
Against Gemini 3.8 Flash, Google reports Hydrogen finishes tasks 2.5x faster at its default setting, scores 57% versus 18.2% on Terminal-Bench 4.1, and makes more than 5x fewer fabricated claims.
Hydrogen batches file-read and search calls in parallel instead of inspecting files one by one, Google says. In early partner testing on graph, SQL and data-heavy enterprise workflows, it ran 2x to 2.8x faster with 12% to 23% fewer tokens than 3.8 Flash.
Google's safety figures versus 3.8 Flash: up to 81% less overreach in agentic tasks, 12x fewer misaligned actions or task shortcuts in tool-use and coding evaluations, and more than 2.3x more resilience on the Gray Swan indirect prompt injection benchmark.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
prompt injection — An attack that hides instructions in content an AI will read — a webpage, email, or document — tricking it into following the attacker instead of the user.
red teaming — Deliberately attacking your own AI system to find what makes it fail before someone else does.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.