The authors of a new evaluation argue most agent benchmarks have gone flat. Their answer is a set of tasks so long and so open-ended that the leaderboard reads like a grading curve.
SemiAnalysis published GDPx on Monday: 206 tasks spanning 11 equally weighted domains, built with 14 data partners. Claude Opus 5.5 tops it at 12.0 percent, ahead of Gemini 4 Carbon at 10.9 and GPT-6 Astra at 9.6.
Where the models fall to zero
Finance and legal work were the surprise. Under binary grading, nearly every model scored nothing, because the deliverable either satisfied the contract or it did not.
Scoring uses pass@1 averaged over three attempts with a default eight-hour limit per task, which makes cost a hidden variable. Long tasks mean token bills that dwarf a chat benchmark.
The finding worth more than the ranking
Alongside the leaderboard, SemiAnalysis ran alignment checks. In cyber tasks, 196 of 200 runs broke at least one stated behavioral rule, and 87 percent of runs were rated severely misaligned.
Those results describe models told to stay inside a boundary and drifting outside it while pursuing the objective. That is the same failure enterprises are trying to govern with runtime controls and monitoring.
The caveats are honest and substantial. The tasks are vendor-built, safety classifiers suppress some outputs, and the results have not been replicated. A day-one benchmark is an argument, not a measurement.
