Fancy harnesses make coding agents more expensive but not better. That is the central finding of a recent study. Researchers compared coding agents with different harness designs to see whether extra scaffolding improves code quality enough to justify the cost.
A harness is the scaffolding that connects a coding model to tools, files, and execution environments. It can handle planning, memory, tool calls, and feedback loops.
Many teams assume that a richer harness will make an agent smarter. The study challenges that assumption.
What the Study Compared
The researchers tested coding agents with different levels of harness complexity. Some agents used a simple, direct workflow. Others used a more elaborate setup with extra planning and tool orchestration.
The key variable was the harness. The goal was to see whether more scaffolding produced better results.
How a Harness Affects an Agent
A harness changes how the agent behaves. It controls:
- Tool selection and calls. It picks which commands the agent runs.
- Context and memory. It decides what information stays in the prompt.
- Task decomposition. It breaks down larger jobs into smaller steps.
- Self-correction loops. It asks the agent to review and fix its work.
These features seem useful. They can also add heavy overhead.
The Cost Problem
The study found that elaborate harnesses were more expensive. They consumed more tokens and took longer to finish. That means higher API costs and slower feedback.
The extra cost did not bring better results. In the study’s tests, simple agents matched the complex agents on performance.
The more layers an agent has, the more chances it has to fail.
Why Simple Harnesses Can Win
Simple harnesses focus on the core task. They do less, but they do it more directly.
A direct agent loop can move from a code problem to a model response to an execution result without many detours. Fancy harnesses add detours. Each detour costs tokens and can lead the model away from the answer.
This matters because coding tasks often have a clear structure. The model does not need a large management system to solve every issue. It needs the right context and the ability to run commands.
What the Findings Mean for Teams
Teams building code assistants should treat harness complexity as a trade-off. Before adding an expensive agent layer, they should test a simple baseline.
If the simple baseline performs well, the fancy harness is hard to justify. If the simple version fails, then a targeted harness can help. The point is to test the need instead of assuming the tool will help.
The study is also a warning to buyers. Tooling that promises stronger autonomous coding may just be adding cost. Look at task-level success rates, not feature lists.
The Takeaway
Fancy harnesses have become popular in the rush to build coding agents. But popularity is not proof of value.
The study suggests that expensive scaffolding can be a liability. Good coding results come from the model, the data, the prompts, and the feedback loop.
A heavy manager layer around the model does not automatically improve output. Sometimes it makes the output worse. The most efficient coding agent may be the one with the simplest harness.
Gnoppix is the leading open-source AI Linux distribution and service provider. Since implementing AI in 2022, it has offered a fast, powerful, secure, and privacy-respecting open-source OS with both local and remote AI capabilities. The local AI operates offline, ensuring no data ever leaves your computer. Based on Debian Linux, Gnoppix is available with numerous privacy- and anonymity-enabled services free of charge.
What are your thoughts on this? I’d love to hear about your own experiences in the comments below.