Hard tasks get handed to teams of agents by default. Fresh evaluation work indicates that habit is costly and seldom earns its keep.
Two rounds of testing went into the evals firm Vals AI’s Vibe Code Bench, feeding GPT-6 Sol and Claude Opus 5.5 first solo and then as teams, under medium and maximum reasoning effort.
Team runs ran between 1.8 and 5.1 times the cost of solo work. On quality, just one of four pairings delivered a statistically meaningful gain, GPT-6 Sol at medium effort, up 7.3 points. Neither model budged at maximum effort.
Anthropic found the same ceiling in its own testing. Bolting agents onto Opus 5.5 shrank the quality gains in two of its tests, and pushing from ten agents to 100 hardly shifted scores after 24 hours, though bigger teams did hit a given level faster.
Token consumption climbed with the speed. Fable 5.1 saw its score slip slightly on a knowledge-base task when its team went from 30 agents to 100.
OpenAI researcher Noam Brown made a comparable case on the Dwarkesh Podcast, calling multi-agent setups mostly a speed purchase: four agents finish a task twice as fast at twice the cost, he said.
Task shape decides the payoff too. Mathematics and web research split apart cleanly; novel writing does not, and Brown likened tossing 10,000 agents at one to tossing 10,000 people.
The takeaway: agent count should track how the work is structured, not how hard the problem is.
