Run an agent around the clock and you find out fast that it needs a computer that's awake. Vercel traces "the laptop lid memes and Mac mini demand spike" to exactly that. A closed lid is only the first of seven things that break when an agent leaves the laptop.
The joke hides a real question. A demo works because you're sitting there. Production works when you aren't. We scouted 123 AI engineering posts across Hacker News, Substack, dev.to, Medium and engineering blogs, then read 33 of the most-shared ones closely. Seven lessons came up again and again. Each one comes with its receipt, and the last section turns them into a checklist you can paste into a pull request.
Cut context. Trim the prompt and write state down. Anthropic cut over 80% of Claude Code's system prompt with no measurable loss on its coding evals.
Job-sized tools. One tool per job, not a wrapper per API endpoint, and errors that say what to do next.
Resumable runs. Log the intent before each call and the outcome after, so a crash replays from the log instead of starting over.
Hard caps. The loop stops at a limit you set. A budget email after the fact is not a cap.
Isolation, not clicks. People click through approve buttons. Remove one leg of the lethal trifecta instead (Fig 4).
Traces and evals. Record every turn and grade real cases pass or fail, so you can find the bad run in 20 seconds.
A person aiming it. The loop runs fast, but a person who knows the job owns the goal.
1. Cut the context, don't hoard it
Anthropic removed over 80% of Claude Code's system prompt for its newest models "with no measurable loss on our coding evaluations". Their own explanation is the useful part: "we found that we were overconstraining Claude Code." Examples hurt too. Giving them to a newer model "actually constrains them to a certain exploration space." And when instructions overlap or conflict, the model has to spend thought sorting them out.
So the first job in production is deletion. Read your prompt and ask what each line is still buying you today.
The second job is deciding what your agent remembers. Memory plugins are the obvious answer, and liao.gg describes them flatly: "A memory plugin analyzes your conversations. It generates 1,000 isolated snippets and inserts them into a vector database." Then comes the lottery. Similarity search hands back whichever snippets look closest, and "you don't know which is correct, current, or what's missing." Picture the guy in Memento with a pile of Polaroids and no idea which one is the latest.
The alternative is boring and works. The working agent writes documentation while "the full picture is still in context," and "facts should be rewritten when truth changes. No appending."
Save. The plugin splits every conversation into snippets and keeps them all. The working agent rewrites the page that changed while the full picture is still in its context.
Recall. Similarity search hands back whichever snippets look closest, old or new. The document has no old version to find: the next run just reads it.
Answer. Snippets pile up and conflict. Facts are rewritten when the truth changes, so the document has one current answer.
Long sessions have their own problem. Drew Breunig calls it Context Inertia: the model commits to a hypothesis or goal "and keeps pursuing it after the evidence says to stop or change course." He reports that thinking tokens per turn fell by 75% as sessions grew, while edits got smaller. Compaction didn't cure it. Stalled runs that could forget gained 4.5% to 9.2% over their next 30 proposals.
His fix is the line we'd tape to a monitor: "For long-running tasks, stash your state in the harness and don't be afraid of a reset."
2. Is MCP dead? Tools after the flip-flop
MCP is the protocol for plugging tools into agents, and the reading shows the field arguing about it in public.
Earendil opened with a confession: "If you went to pi.dev in the past, you found a proud declaration that Pi does not support MCP." Now it does, because "the MCP of today is not the MCP of yesteryear." Simon Willison's post on stateless MCP calls the new stateless spec "the most significant change to the MCP spec since it first launched", and servers no longer have to track session IDs.
The other side isn't shy. One post says it's time to end-of-life MCP and use HTTP APIs and command-line tools, since models now figure out --help on their own. Armin Ronacher's Codemode takes a third road: the model writes code against your tools, and that code runs in the harness, in its own sandbox. His reason is that bash "can only compose programs that run."
Four posts, four positions. Only one is still saying no.
| Post | Where it landed | Best line |
|---|---|---|
| Earendil | Reversed, now supports MCP | "the MCP of today is not the MCP of yesteryear" |
| Simon Willison | Interested again, thanks to stateless MCP | "the most significant change to the MCP spec since it first launched" |
| maharship.com | Drop it for APIs and CLIs | "time to end-of-life it" |
| Armin Ronacher | Let the model write code against tools | "Codemode runs in the harness, in its own sandbox." |
Willison adds a practical point for the MCP side: its tools are "easier to audit and control". Earendil admits the cost: "The biggest issue with MCP continues to be that it's hard to compose."
Underneath the protocol fight sits a quieter question: what should one tool look like? Here the posts agree. Luca Rossi's guide to designing an MCP server calls the most common mistake "taking the endpoints you already have and turning each of them into a tool." Design around the job instead. Notion's fix was to let pages move in and out as Markdown, so "one page-level call replaces a chain of block calls."
Two more rules from the same post. Tool descriptions behave like code: "changing a sentence there can change what the model does as much as changing code." And errors should say what to do next, because an agent "benefits from errors that are more descriptive."
The protocol is the easy argument. The tool is the hard one.
3. Runs have to survive a closed lid
Say your agent is halfway through a 40-minute task when the process dies. What happens next? Vercel is blunt that a VM alone isn't the answer: it "doesn't automatically record which model and tool calls completed or resume interrupted turns after a crash."
Their durable-harness fix is a log. "Before a model call or tool call runs, it records an intent, and when the call finishes, it records the outcome." After a crash, the run replays from the log. Finished calls aren't repeated, since their results are read back, "so you're never billed twice for a finished model call's tokens."
Then they split tools in two. A lookup is safe: it "can run again after a crash." A "never" tool, such as a refund, never runs twice.
- Duration
- 31s
- Log entries
- 5
- Outcomes missing
- 0
- Refunds sent
- 1
Log. Before a model or tool call runs, the run records an intent. When the call finishes, it records the outcome.
Crash. The worker dies halfway through the refund. Its intent is in the log; its outcome isn't.
Replay. Finished calls are read back from the log, so you're never billed twice. The refund is a “never” tool, so it isn't run again: its status gets checked instead.
Sort tools by what a repeat costs. A lookup can run again after a crash. A refund never runs twice.
If your agent writes to the outside world, the same idea has an older name. Rossi points to Stripe's idempotency key: "a retry with the same key returns the first response instead of creating a second object." Give every write a key, and a retry stops being a double charge.
4. Put a hard cap on the bill
Simon Willison's post on hard budget caps says it all: "We're going to need default hard budget caps on pretty much everything." His reasoning is short. "Soft caps, 'after $X/month, send me a warning email', will not cut it." He expects that "most businesses and individuals would prefer errors to a surprise $10,000+ bill." The clouds are catching on. AWS can pause a project at its spend limit, and Google Cloud launched a similar feature, Spend Caps, in July.
An agent's bill can double without a single architectural change. Cloudflare's code reviewer is the example. In April an average review cost a palatable $1.19. By the end of August it was about $2.46. Their data showed the first request of a session found a cached prefix only 14% to 26% of the time. Fixing that was the biggest single saving, and with model and effort changes on top the average review now costs $0.77, "down 68% from the $2.43 it averaged over our two most expensive weeks before we started."
| When | Cost per review | What changed |
|---|---|---|
| April | $1.19 average | Starting point |
| End of August | About $2.46 average | No significant architectural changes |
| Two most expensive weeks | $2.43 average | Baseline for the 68% cut |
| After the work | $0.77 average | Down 68% from $2.43 |
The cheaper version wasn't worse. The median review dropped from 4m 09s to 1m 32s, and authors acted on 59% of the code quality reviewer's serious findings instead of 34%.
Model choice moves the bill even more. Cursor tried several model mixes on an agent swarm: "Every mix produced similar quality, but the costs varied enormously", from $1,339 for the Opus 4.8 hybrid to $10,565 for GPT-5.5 alone.
Cost is a bug you can ship. A hard cap makes it fail loudly instead of quietly.
5. Approve is not a firewall
The usual safety plan is a click: the agent asks, a person approves. Scalex built a game that tests it, and the average player missed 1 in 3 threats. A game isn't an incident report, but the pattern is telling. Hiding a payload behind a familiar script name roughly doubled its success rate, even when the payload was visible in the history log. The most-missed command, npm run analyze, was approved 64.7% of the time. And people got worse near the end: miss rates climbed back up after a short warm-up.
An automatic approver doesn't close the gap either. Embrace The Red hijacked Claude Code in Auto Mode with a plain website-summary request, with attack success rates up to 80% on a small sample. The line to remember: "So 0.00% on the benchmark and a working RCE are both true at once." Their advice is that Auto Mode is not a substitute for an isolated environment.
Now the part that surprises people. The pages your agent reads talk to it. Redact read all 545 pages that WhatsApp's help centre links to. Most of the 102 blocks they found were harmless, but until mid-October 2026 some told an AI not to recommend the regulator. Nobody saw them on screen. They were there for an AI to read.
This is an old bug in a new place. As a dev.to post on prompt injection as the new SQL injection says, "Prompt injection is that exact flaw, moved up a layer." Little Bobby Tables made SQL injection a joke every developer gets, and parameterized queries made it fixable. Natural language has no such fix yet: "we don't have the 'parameterized query' equivalent."
So what do you do? Count legs. An agent is exploitable when it has access to private data, exposure to untrusted content, and the ability to communicate externally. Simon Willison's name for that is the Lethal Trifecta, and the dev.to post's advice is simple: "Remove any one leg and the exploit loses its teeth."
All three legs. Private data, untrusted content and a way out together are the exploit: a page the agent reads can tell it to send your data, and it can.
No way out. It still reads your data and the poisoned page, but it has nothing to send with.
No untrusted content. Only your own material goes in, so nobody outside can slip it instructions.
No private data. A poisoned page can still steer it, but there's nothing worth stealing. Remove any one leg and the exploit loses its teeth.
Ethan Mollick's essay on agency and agents adds the human half. He lists at least four situations where agents should seek human help, and his verdict is "We need agents that know when to look up." You want your agent asking a person for the right things, not for everything.
6. You can't fix what you can't see
Everything above assumes you can look at what happened. One engineer's argument for running your own LLM proxy starts from the fact that a provider dashboard is "not token usage statistics with time series data meant for engineers." With a proxy you get logs of "which key called what model with what client at what time and its status", and a cache miss in a coding session becomes a 20-second hunt.
A trace tells you what happened. An eval tells you whether it was any good. The evals FAQ by Hamel Husain and Shreya Shankar is the clearest rulebook we read, and we'd paraphrase it as five rules:
- Spend the time on errors. In the projects they've worked on, they've spent 60-80% of development time on error analysis and evaluation.
- Score pass or fail. "Binary evaluations force clearer thinking and more consistent labeling."
- Skip generic metrics. BERTScore, ROUGE and cosine similarity "are not useful" for most LLM applications.
- Don't write evals before you've seen errors. Eval-driven development "creates more problems than it solves."
- Build your own annotation tool. In Hamel's experience, teams with custom annotation tools "iterate ~10x faster."
Vercel's durable harness gets a small bonus here: turns that resumed after a crash show up in the same trace, so you can see where a turn stopped and picked back up. We wrote a walkthrough of the same loop, how to ship an agent change without breaking it.
7. The expert is still the multiplier
After so much about harnesses, a human point. Sean Goedecke's essay on LLMs rewarding expertise argues that "the most important skill in prompting is expertise in the domain you're prompting for." For many tasks, "the human is the bottleneck, not the model". He even concedes, "Terence Tao is a better mathematician than I am a programmer." The point holds anyway: the better you know the domain, the better you steer.
Ethan Mollick's write-up of OpenAI's agent swarm agrees. Thousands of agents sent about 2.7 million messages and reached their result after 88 hours, yet "the agents did the organizing but people decided where to point them."
Agents multiply what you bring. Bring a vague goal, and you get a very busy vague goal.
The checklist
Here are the seven lessons as questions. Paste them into the pull request that moves an agent to production.
| Lesson | The check | Ask yourself |
|---|---|---|
| Context | Prompt trimmed, state written down | What would I delete without a regret? |
| Tools | One tool per job, errors say what's next | Did I wrap endpoints or jobs? |
| Runs | Intent and outcome logged, writes keyed | If it dies now, what runs twice? |
| Bill | A hard cap, not an email | What stops a runaway run? |
| Safety | One leg of the trifecta removed | What if a page I read lies? |
| Visibility | Traces plus pass/fail evals | Can I find the bad run in 20 seconds? |
| Expert | A person owns the goal | Who decides where this points? |
We build Definable around a few of these. Every agent run has a trace with each model call, tool call and the moments it waited on a person, publishing can run every eval case on the draft first, and risky tools can sit under "Needs approval".
Find the row you'd fail today and fix that one first. Then the lid can stay closed.