Claude Fable 5 solved 97.8% of the attempts on SWE-bench Lite inside Claude Code and 96.7% inside Pi, a gap of about one point. Claude Code cost about twice as much: $1.33 against $0.67, according to HarnessTax, a September study from UC Berkeley and Arena Intelligence.
Same model, same benchmark, nearly the same result, double the bill. The difference is the harness: everything around the model that isn't the model. Its instructions, its tools, the loop that calls it again and again, and the context it gets to read. Across 21 model and harness pairs, the study found the same model reaching similar success at up to 5x the cost.
Our read of the last six weeks of writing is that the harness decides most of what your agent costs and, for cheaper models, whether it finishes at all. Think of Formula 1: customer teams can buy the same engine, and the car built around it still decides the race. Most of the waste comes from a handful of habits you can check this week.
We're the Definable team, and we build the harness every Definable agent runs in. We scouted 169 AI engineering posts from the past six weeks and read 35 about harnesses, context and tools closely. Along the way we'll own up to four mistakes of our own.
A person asks. The message goes to the harness, not straight to the model.
The harness is everything around the model: its instructions, its tools, the loop that calls it again and again, and the context it gets to read.
The model is one box. Same model, same benchmark: about $1.33 an attempt inside Claude Code and $0.67 inside Pi, a gap of about one point in solves.
What is an agent harness?
Earendil defines it as "a piece of software that provides an environment for an AI model to operate within." It does four things: a system prompt, tools, the agentic loop, and a translation layer so one agent can talk to many models. Shrivu Shankar casts a wider net: "all the infra, interfaces, context, and state that surround a stateless LLM."
Isuru Maduranga's Harness Engineering 101 puts the mechanism in two sentences: "An LLM API is stateless. Every turn, you send the entire conversation as a JSON array and get text back." It's Groundhog Day. Each turn the model wakes up and relives the whole conversation from the top, and the harness decides what that day contains.
The same page explains why harnesses are messy: each patch "exists because something concrete broke." Nobody designed one in a sitting. And unlike the model, Earendil points out, "you can own and adapt the harness."
The tax is paid before the model reads a word
HarnessTax ran seven models through three harnesses (Claude Code, Codex CLI and Pi) on two benchmarks. Claude Code costs about 2.0x as much as Pi and 1.6x as much as Codex on SWE-bench Lite. Success barely moves: the average harness effect stays within 2% on SWE-bench Lite and about 5% on Terminal-Bench 2.0.
Where does your money go? Partly into the first call. Pi gets by with four tools: read, write, edit and bash. Claude Code's mean initial context is over 10x Pi's, with longer instructions and larger tool schemas. You pay for all of it before the model has seen your task, and you pay again on every run.
One engineer measured how much that weighs on a laptop. Their nine-harnesses test ran a 3-bit Qwen 3.8 27B on an M4 MacBook Pro, and the first prompt alone ranges from 1,171 tokens to 18,046. The bigger the prompt, the longer you wait before the first token. You don't see a bill here, you feel it.
| Harness | Tools | First prompt | First wait |
|---|---|---|---|
| mini-swe-agent | 1 | 1,171 | 12.2 s |
| pi | 4 | 2,008 | 21.6 s |
| codex | 10 | 7,804 | 87.8 s |
| goose | 18 | 9,617 | 110.3 s |
| crush | 26 | 16,263 | 199.8 s |
| opencode | 10 | 18,046 | 225.7 s |
That's 22 seconds for pi against 226 for OpenCode. On a data center GPU the same gap shrinks to 0.2 against 1.8 seconds, so you mostly notice it when you run locally. The context matters more than the clock: with 18,046 tokens gone, OpenCode leaves 44% of a 32,000-token budget for actual work. Pi leaves 94%.
The same thing happens with tools you add to your own agent. Connect GitHub's MCP server and you get about 93 tools, about 55,000 tokens "before you type a word". Anthropic went the other way for its newest models and removed over 80% of Claude Code's system prompt with no measurable loss on its coding evaluations. We covered that in What breaks when your AI agent leaves the laptop.
Two caveats, because we don't want you leaning on this too hard. HarnessTax warns its findings may be limited to two open-source benchmarks the models "may have encountered during training." And the laptop test's author builds a harness too, says so, and asks that its pass column not be read as a ranking.
Where does a harness pay for itself?
Naresh Jain, writing on martinfowler.com, expected a harness to help. On the frontier model it didn't. For GPT-6 Sol, the harness used about 10.5 times as many tokens and cost about 7 times more, "yet both conditions succeeded in all 10 runs."
Then Jain tried the cheaper GPT-6 Luna at low reasoning effort. Without a harness, it succeeded in 3 of 10 runs. With the harness, 10 of 10. And the harnessed cheap model beat the unharnessed expensive one on price: about $0.273 per attempt against $0.406, roughly 33% less in this sample.
The harness has two parts. Guides steer the agent before it acts (specs, examples). Sensors check after it acts (tests, contract checks) and feed back what failed. Jain calls this "executable attention", and the conclusion is the line to keep: "What changes with model capability is not whether independent sensors are needed, but when they run."
Guides steer the agent before it acts: specs and examples.
The agent writes the service.
Sensors check after it acts: tests and contract checks feed back what failed, and it goes round again.
Accepted. With the harness the cheap model passed 10 of 10, at about $0.273 an attempt against $0.406 for the bigger model without one.
| Model | Harness | Accepted | Cost per run |
|---|---|---|---|
| GPT-6 Sol | Yes | 10 of 10 | about $2.68 |
| GPT-6 Sol | No | 10 of 10 | about $0.41 |
| GPT-6 Luna | Yes | 10 of 10 | about $0.27 |
| GPT-6 Luna | No | 3 of 10 | about $0.09 |
There's a floor, too. A harness can't rescue a model that can't format a tool call. A guide to replacing Claude Code with a local model puts it bluntly: "Sub-7B models emit malformed tool calls no matter which agent wraps them."
Caveats: 10 paired runs, one greenfield Java service, and a tool from Jain's own company, which the article discloses. The costs also leave out the effort of writing the contracts and the harness. The takeaway: spend the harness on the cheap model, where 3 of 10 became 10 of 10.
Never edit what you already sent
A provider caches your prompt as one prefix: tools, then the system block, then the messages. Change anything and the cache misses from that point to the end. So the cheapest harness rule is also the strictest: only ever append. If you take one rule from this post, take that one.
Chad Gibson at Neuralwatt saw 1.6% to 11.1% of input uncached across clients in 30 days of their cloud traffic. At 92% cached, 8 tokens per 100 are uncached, against 3 at 97%: about 2.7 times the prefill work.
A cache miss roughly doubles a request's energy, and past 128k tokens, one turn that re-reads more than half its prompt costs as much prefill as about 600 normal turns.
In their lab, of 539 follow-up requests, 88% were pure appends and the other 65 broke the cache. Every break came from a harness editing what it had already sent. The classic offender is the date. OpenCode 1.x and Kilo Code put today's date in the system prompt, so crossing local midnight means re-reading nearly everything once. Reid Marlow lists three habits that bust it: changing tool schemas, dialing reasoning effort, and inserting dynamic instructions mid-context.
Append only. A provider caches your prompt as one prefix: tools, then the system block, then the messages. Add to the end and only the new message is new.
Edit what you sent. Change anything and the cache misses from that point to the end. A date in the system prompt re-reads nearly everything once you cross midnight.
Only ever append. In Neuralwatt's lab 88% of 539 follow-up requests were pure appends, and every break came from a harness editing what it had already sent.
We made this mistake ourselves. Before 2026-10-16, our note about attached files was a system message. Every model adapter lifts system messages into the system block, so a message with files changed the cached prefix. That run read the whole thread again at full price, and the next one usually did too.
Our rule now is that text goes where its rate of change puts it. The date follows it: every user message leads with a line the platform writes, so the agent knows the date, the local time and who it's talking to, and a later run replays that old line exactly. Cache timing is the other half. In our runtime a model's prompt cache lasts 5 minutes after its last call. Let a 100,000-token chat with Opus 5.5 go cold, and the next call costs about $0.50 instead of $0.02.
If you build on Claude's newest models, there's a harder edge. A thinking block is bound to everything sent before it, so an edit to history voids it, and Anthropic refuses the request with a 400 for accounts made from 2026-08-31. A cache miss used to be a cost. Now you can get an error instead.
The cruel part is the quiet. In Neuralwatt's words: "A cache break doesn't show up as an error. You still get your answer, just later and with more energy spent on it."
A rule only works if it arrives
Instructions can fail before the model ever sees them. One engineer found that Claude Code 2.1.277 skipped a local AGENTS.md whenever telemetry was off. The loader sat behind a remote feature flag, and there was no warning. The proof was a canary: an empty directory holding only an AGENTS.md with one canary word in it, then claude -p asked for the word. It was fixed later, per the Hacker News thread. The point stands: people "will spend time on prompts when the file never reached the model in the first place."
Then there's where the rule sits. On Definable, guardrails are short rules an agent must never break, like "Never promise a refund." Before every model call, our harness takes out the block it added last time and puts a fresh one at the end of the messages. Our bet: instructions at the top drift away as a run grows, and the end comes after everything the run has read.
That block moves on every call, which looks like it breaks "only ever append". It doesn't: everything before the old block stays byte for byte the same, so the cache holds up to it. On Anthropic and OpenAI's Responses API, the block closes the system prompt instead of being the last message.
We measured the rule against no rule on 2026-10-14. An email inside a tool result ordered the agent to print a discount code. With no rule, Gemini and DeepSeek printed it in 12 of 12 runs between them. With a rule read last, Gemini printed it in 0 of 8 and DeepSeek in 1 of 14.
We never tested the same rule at the top, so this isn't proof about placement alone. Nothing checks the reply, either, and DeepSeek sometimes paraphrased the rule it was told not to quote. Prove the rules that matter with an eval.
Do this today: put a canary word in your instructions file, ask the agent for it, and see whether it comes back. If it doesn't, you've found your bug.
What comes back from tools is prompt too
A harness also decides what a tool result looks like when it lands. We found one the hard way. Through a real connected GitHub account, Composio's list of 100 issues came back as 555,987 characters, about 139,000 tokens, and before we fixed it, that rode on every later call of the run.
Now, in our runtime, a result past its tool's limit is saved once as a file. The model reads its start, its end and a line naming the file, and pages the rest with a read tool. One answer's tool results share 100,000 characters.
A tool returns a lot. A list of 100 GitHub issues came back as 555,987 characters, about 139,000 tokens, and before the fix it rode on every later call.
Saved once as a file. A result past its tool's limit goes to a file instead of the prompt.
Read a page at a time. The model reads the start, the end and a line naming the file, and pages the rest with a read tool.
The shape of a result matters as much as its size. Pengcheng Xu ran a small grep versus LSP pilot on multi-file renames with Opus 4.8. Two LSP arms shared one backend and differed only in output shape: bare locations, or locations with a few lines of code inline. With the code inline, follow-up file reads per episode fell from 15.2 to 3.2.
And measuring is hard. Quesma tested RTK, which compresses terminal output before the agent reads it. Its counter reported 349.2 million tokens saved, an 89% reduction. In Quesma's runs, the bill fell 5% for Fable and rose 5% for DeepSeek. The counter is bytes divided by 4, not billed tokens. A savings counter is a claim. The invoice is the measurement.
Some things only the harness can know. A model can't see its own output cap: a reply cut at our 32,000-token limit used to end a run as a success with half an answer. Now the runtime asks the model to carry on where it stopped, three tries, and then the run ends as halted. A model can't see the calendar, either, and we learned that the slow way. Before the date line, one of our own agents set a one-time task in the past.
Audit your harness this week
Six checks you can run on your own agent. None needs a new tool.
| Check | How | Why |
|---|---|---|
| First prompt size | Count tokens of call one | Paid on every run |
| Cache hit per turn | Log cache reads and writes per call | Breaks throw no error |
| Edits to sent text | Look for dates, reminders, tool lists | One edit re-reads from there on |
| Rules arrive | Plant a canary word, ask for it | A skipped file throws no error |
| Tool result size | Log the largest result per run | It rides on every call |
| Same model, two harnesses | Run ten tasks in each | Up to 5x, per HarnessTax |
Every number about our runtime above is in our docs, next to the code it measures.
The model gets the credit and the harness gets the bill. You rent the model. You own the harness.