Context is king: how to cut AI agent token costs

An agent loop looks like one continuous conversation. It is not. The Claude Messages API is stateless: the full conversational history goes to the API on every request. Step forty does not send step forty. It sends steps one through forty, and pays for all of them at input rates.

That is the shape of the curve, and it is why the number surprises people watching the wrong variable. Cost tracks the length of the run rather than the difficulty of the work in front of it, because the work in front of it is the smallest thing being transmitted.

The scale has been measured outside anyone's product. A study of agent trajectories from eight frontier models on SWE-bench Verified found that agentic coding tasks consumed roughly 1,000 times the tokens of code reasoning and code chat, with input tokens rather than output tokens driving the overall cost. The domain is narrow and worth naming: that is agentic coding, measured against chat and reasoning baselines, and not a comparison between one agent setup and another.

Prompt caching is the standard answer, and it is a real answer to a smaller question. Caching lowers what a repeated prefix costs. It does not stop the prefix from being sent, it does not make the history shorter, and it does not change which step is carrying the weight.

The rest of the usual list behaves the same way. Routing cheap steps to a smaller model, pruning tool definitions, trimming verbosity, batching: each one makes the transport cheaper, and not one of them changes what is being transported. Do all of it. Then notice that the curve still bends with the length of the run, because what drives it was never the price per token.

Sending fewer tokens is the obvious move and the wrong question. The same study measured what the extra tokens bought, and the answer was nothing: higher token usage did not translate into higher accuracy, and accuracy often peaked at intermediate cost and saturated at higher costs. Spending more did not start failing. It stopped mattering.

That splits a bill into two parts which behave differently. One part buys quality, and it is the part where the model is given something it did not already have. The other part only buys re-transmission, and its size is set by how long the run has been going.

A second finding is worth carrying forward, because it gets used against my own number later. In the same study, runs on the same task differed by up to 30 times in total tokens, and the frontier models systematically underestimated their own token usage. Agent runs are not tightly distributed, and neither is any single reading taken from one.

Put the mechanism and the saturation result side by side and the reasoning is short. Every step re-sends what came before it, and past a point those tokens stop converting into accuracy. So a share of a long run's bill goes on re-transmitting state that has already done its work — the exact spend that moving state into named, re-readable artifacts removes. How large that share is, I cannot tell you. Nothing I found measures the overlap directly, and a widely repeated figure is one I could not trace to a primary source, so it is not going in here.

The natural objection is that this is a capacity problem and capacity keeps improving. It has improved. A 2026 evaluation of 26 models across a fixed 8K to 1M token grid found long-context quality to be length-dependent rather than a single property. Rankings reshuffled between the 8K-128K and 8K-1M aggregates: 7 models moved at least two ranks, and individual rank gaps reached 12 positions.

Read that as a benchmark-design result, because that is what it is. It does not say performance collapses somewhere, and it offers no account of why. What it says is that the leaderboard you picked your model from was measured at a length, and your long run is not at that length.

Compaction is the other half of the reply, and it is a trade rather than a fix. In September 2025, Anthropic's applied AI team described context as "a finite resource with diminishing marginal returns", warned that aggressive compaction can lose critical context, and recommended that agents write notes persisted to memory outside the context window and pull them back in when they are needed. Compaction buys room by deciding what stops existing, and it makes that decision without consulting the stage that will need it. The last of those recommendations is the architecture the rest of this article is about, argued by the people who build the model.

I did not design this. I work with 10x-devs, a skills-based, file-based agentic development methodology shared with attendees of the 10x-devs 3.0 course, and I applied it to a project of my own, open_offensive. The architecture came from there, and what follows describes it rather than claims it. Structure as an engineering decision is not a new move either; it is how development methodologies got their structure.

The rule is one sentence. A stage writes its output to a named file, and the next stage reads that file instead of the conversation that produced it.

In open_offensive, the tree under context/ holds 141 files as I write this. foundation/ carries the 7 documents that do not change per unit of work — among them the product definition, the roadmap, the tech stack, the test plan, the infrastructure notes. Each change gets a folder of its own, with a change.md stating what is being built, a plan-brief.md, a plan.md, a research.md where research was needed, and a reviews/ directory holding the review of the implementation. Finished work moves to context/archive/, which holds 25 change folders dated 2026-07-06 to 2026-08-09.

The objection arrives immediately: the agent reads those files back into the window, so the tokens come back. They do, and it is not the same transaction. A file is read once, by the stage that needs it. A conversation is re-sent once per step. One is a cost you pay when the information is required, the other is a cost you pay because the information exists.

The property doing the work is rereadability. A file re-reads identically after a cleared session. A compacted history does not, because what compaction dropped is gone and the stage that needed it cannot ask for it back.

One distinction before this gets confused with its neighbour: it is state for a workflow, not retrieval for knowledge. A vector store answers what the corpus says about a topic. An artifact tree answers what stage three decided, and it has to answer identically every time it is asked.

Before this goes further: the architecture has been published independently, by people I have never met.

In March 2026, Jake Van Clief and David McDermott posted a paper to arXiv proposing what they call the Model Workspace Protocol. It replaces framework-level orchestration with filesystem structure. Numbered folders represent the workflow stages, plain markdown files carry the prompts and context that tell a single agent what role to play at each step, and local scripts handle the mechanical work that does not need a model at all.

That is the same design, not a similar one.

I am not claiming priority over it and I am not conceding any either. The dates of my own builds are not something I documented, and a priority argument would be the least interesting thing here. Two parties reaching the same structure separately is the most ordinary kind of evidence that the structure is right. If this were only my preference, it would be a coincidence that someone else's preference has the same folders in it.

There is a practical reason to put the paper here rather than in a footnote. A reader who goes looking will find it, and finding it in somebody else's article is a different experience from finding it in mine.

What the paper does not carry is a measurement: it reports no quantitative results at all. It also does not test whether the design survives being moved into a domain it was not built for. Those two absences are the next two sections.

I built 10x-Pentest, a context-engineering methodology for penetration testing delivered as agent skills, in which stages communicate through files rather than session memory. The repository holds 20 skills as it stands today, covering scope, recon, threat modelling, source-code analysis, dynamic testing, review and reporting. I have run a complete engagement for a real client through it, from scope to delivered report. The client's name stays out of this article.

Penetration testing was the first domain I moved, because it was already a staged process long before automation touched it — the standards a pentest is bound by name the phases for you. What a staged engagement leaves behind as an audit trail is a separate subject, argued in what a staged, file-based engagement leaves behind; here the point is the bill.

The number comes from a separate exercise. One web application, tested twice: first ad-hoc with agentic AI, then again through the staged workflow. The staged run consumed 30% of the tokens the ad-hoc run consumed.

Now the part a before-and-after post usually leaves out.

The framework run was second, on a target that had already been explored once. Prior knowledge of an application reduces token spend on its own, with no workflow involved. This is not a controlled A/B and I will not present it as one. It is one paired run with a confound I can name.

There is a sharper reason to hold it loosely, sitting two sections up. Runs on the same task have been measured differing by up to thirtyfold in total tokens. Against variance that wide, a single paired comparison is an instance rather than a measurement, which is why the argument here rests on the mechanism and the saturation result and not on my ratio.

I keep the ratio because it is what happened, and because a number withheld is worth less than a number with its limits printed beside it.

One architecture, three domains, one operator. Software development, in open_offensive, through 10x-devs. Penetration testing, in 10x-Pentest. Content production, in 10x-Seo. All three keep stage state in files under a context/ tree rather than in the session.

Put two of the trees side by side and the resemblance is structural, not cosmetic. 10x-Pentest keeps its durable documents in context/foundation/ — the scope schema, the finding schema, the report template, the tool catalogue — and its stages read those files and write named outputs into the engagement's folder. Swap the nouns and it is the development tree. The stage names differ because the work differs. The handoff does not.

The direction matters, and it is the opposite of a discovery story. 10x-Pentest and 10x-Seo were derived from 10x-devs. I did not arrive at the same design three times. I carried one design into two domains it was not built for, and it held. How much I copied and how much I redesigned is not something I tracked, and "it held" is the whole of what I will claim — not that it held easily.

What transfers is the architecture, not the prompts. A pentest scope document and a software change brief have nothing in common. They have the same job: to be what the next stage reads instead of what it is expected to remember.

Then the limit, which is permanent. I wrote two of the three. One operator applying his own design to his own projects is not evidence that the design is operator-independent. This is n = 1, and it is permanent: a fourth tree from the same hands is the same n. That objection cannot be answered by more of my work. It can be answered by a test you run on yours.

Open your own workflow and walk it stage by stage. At each one, ask a single question: what does this stage need that exists only in the session?

Not what it reads. What it remembers.

Load-bearing state is anything a later step depends on — a decision and the reason behind it, a constraint discovered halfway through, a rejected option and why, the agreed shape of the thing being produced. When it exists only as something said earlier in the conversation, it is carried by re-transmission, and re-transmission is the part of the bill that stopped buying accuracy.

Here is what the question catches. A stage writing the final summary needs to know which options were rejected earlier and why, or it will quietly re-propose them. If that reasoning exists only in the review conversation, the summary stage is one cleared session away from being wrong, and no prompt improvement repairs it.

You will find the leak fastest at the stage whose instructions amount to "continue from where we left off". It is rarely the first stage. It is usually one in the middle that inherited a decision no one wrote down.

The artifact that replaces the remembering has three properties, and each is doing work. It is named, so a stage asks for scope.md rather than for what was decided about scope. It is complete on its own, so reading it does not require having been there when it was written. It has a shape the next stage can rely on, so the handoff can be checked instead of hoped for.

With those in place the handoff becomes an instruction: this stage reads these files, and writes that one. The conversation becomes where work happens rather than where results are stored, and it can be thrown away at any point without taking the run with it.

The test takes about a minute, and it does not ask you to trust anything above.

Start a run. Get into the middle of it, past the point where real decisions have been made. Clear the session. Then continue.

A pass is boring to watch. The next stage reads its inputs from files, produces what it would have produced anyway, and the clear costs you the time to re-read them. A failure is unmistakable. The work stalls, or it carries on against a drifted version of the decisions, because whatever the run needed and only the conversation held is gone.

Compaction is the same test with a softer edge. Anthropic's applied AI team warned in September 2025 that aggressive compaction can lose critical context, and the difficulty is not the loss but that nothing tells you which part was lost. A file re-reads identically. A compacted history hands back a summary of itself, and there is nothing to diff it against.

One word to be careful with. This makes a run recoverable, not reproducible. No pipeline built on a language model is run-identical, and moving state to disk does not change that. What changes is that the second attempt starts from the same place as the first.

I have applied this architecture across three domains, and I am deliberately not reporting a result for this test in any of them. A test you run in a minute on your own workflow does not need my outcome, which is why I am offering the test rather than the anecdote.

The overhead is real and most of it is writing. Every stage that hands off has to produce a document worth reading, and every document has to stay true as what it describes changes. A schema that has drifted from what stages actually write is worse than none, because it is still trusted. The tree grows, too: open_offensive sits at 141 files and 25 archived change folders, and nothing has been deleted, which is a decision with a price. It is the same work in all three of my trees — the maintenance cost nobody budgets for, in a different register.

There is a case where this is the wrong structure. Short work with no handoff and nothing to hand over belongs in the session. If the run is one stage, if it finishes before the history gets long, and if no one has to reconstruct how it got there, the files are ceremony and the session is the right place for state.

The line is not size. It is whether anything downstream — another stage, another person, or you in three weeks — has to read what this run decided. When something does, the session is the wrong place to keep it, and the bill is only the part of that you can see.