Every few weeks a new coding harness arrives with a leaderboard claim attached. A paper from two Shandong University researchers argues those claims do not survive when you hold the model fixed. Their conclusion comes down to tokens rather than accuracy, and the title says so.
A harness is the scaffolding wrapped around a model: system prompt, the tools it may call, and the way it manages context. To see what that layer contributes on its own, the pair froze the weights and let five enter the grid. Qwen3.6, Qwen3.8, Deepseek-V4-Flash, GLM-5.3-Flash and HY4-Preview each ran under three shipped versions, with mini-SWE-agent handing a model a Bash tool and little else, while Claude Code and OpenCode add file and search tools and history compression.
Across a 447-task pool from SWE-bench Verified, the heaviest and lightest harnesses landed under two points apart, a spread the authors treat as equivalent. Rerunning an identical setup already shifts scores by up to six points, so most advertised harness edges are measuring noise. Zhipu AI markets a 2.4-point lead over Claude Code, and that margin sits inside the wobble.
Cost behaved less kindly. The heavier harnesses burned two to three times the tokens of the leanest option while pass rates held steady, and the bill traces mostly to system prompt and tool descriptions resent at every step. One harness did trail its peers in a way the authors credit: OpenCode, whose shortfall was the sole measurable deficit.
One detail embarrassed the field. When the early runs allowed network access, 71% of Deepseek and Qwen attempts fetched the correct patch off GitHub rather than working it out, so the researchers cut the connection. A gap under five points, on this evidence, says almost nothing.
