Introducing Physical Engineering S Last Exam Ple With Google Cloud Ai Research

Physical-Engineering's Last Exam, or PLE, is a new benchmark from Mercor, a company that builds AI evaluations with a network of domain experts, and the AI and infrastructure research team at Google. It tests whether AI agents can do the design work engineers sign off on, and its headline result is that they cannot yet be fully trusted to. According to Mercor's announcement, the best agent configuration, Claude Opus 5.5 running in Claude Code, scored 48.3% on the hidden test set. Its designs met 91% of individual constraints on average, but only 41% cleared every hard safety and feasibility check.

That gap is what the benchmark is built to expose. An engineering design that gets most things right still fails review if one load calculation or code requirement is wrong, so PLE awards points for cost efficiency only after a design passes every hard check. The 85 tasks span mechanical, electrical, civil and structural work, such as sizing a switchboard and its cables, designing a cooling loop or checking a roof deck. Licensed engineers wrote 44 of them end to end. Google generated the other 41 with an adversarial loop in which one agent drafts a task, a second agent tries to break it and the first hardens it, and engineers validated all 85. Each task runs in a project folder of about 61 files, including drawings, 3D building models, client emails and outdated revisions planted as traps. The agent has no internet access and up to six hours, and much of the difficulty is working out which requirements actually govern the design.

Mercor and Google ran 14 frontier models across seven agent harnesses, the software that gives a model tools and runs its work loop. The model mattered far more than the harness. Scores across models spanned up to 50 points, while moving the same model to a different harness changed its score by a median of about 3. Claude Opus 5.5 led, followed by GPT 6 Astra at 45.8% and Gemini 4 Argon at 40.6%. The top configuration used an average of 27 million tokens and 63 minutes per task. The benchmark extends Mercor's APEX-Agents work on office tasks such as banking and legal analysis. Google has released 29 public tasks, the grading tools and the evaluation container, while scored competition runs on the hidden set. The announcement does not give per-discipline scores or the full scoring formula. This explanation draws on the announcement alone, not on the accompanying paper or Google's companion post.