Most agent benchmarks test office work. Google’s AI and Infrastructure team, working with data vendor Mercor, built one around the jobs that produce physical things.
Physical-Engineering’s Last Exam holds 85 long-horizon tasks across mechanical, electrical, civil and structural engineering. This week 29 of them, plus the grading tools and evaluation container, went public alongside an open competition and leaderboard.
What a task looks like
Each runs in a project folder averaging 61 files: PDFs, spreadsheets, correspondence, superseded revisions, and drawings in DXF, DWG or IFC format. The agent must work out which documents govern, then issue deliverables a licensed engineer could review, such as calculation workbooks, equipment schedules and 3D models.
Examples include sizing steel roof decking under snow-drift and shear loads, laying out a phased campus around wetland buffers, and picking heat-rejection equipment against a 25-year cost objective.
Practising engineers wrote 44 tasks end to end. In the other 41, an adversarial pipeline built the environments and graders from real documentation, then experts raised the difficulty.
The scores stay low
On the private 56-task set, no model cleared 50 percent in its native harness. Claude Opus 5.5 in Claude Code led at 48.3 percent, followed by GPT-6 Astra at 45.8 and Claude Sonnet 5.5 at 43.5. Several open models landed in single digits.
The leader spent roughly 27 million tokens and 63 minutes per task. Grading combines code compliance with an economic objective, so a design can pass every check and still lose points for costing more to install.
Google says solvers ran without internet access. The remaining 56 tasks stay hidden for the competition.
