Last week I spent an hour and a half taking a developer on my team through fundamentals: what a token is, what a skill is from the model's side of the glass, what a runbook is, how memory actually works in the tools we use every day. I had it planned — none of this is new to me. What I could not do was point them at anything to read instead, because none of it is written down in one place.
The absence has a shape, and you can see it in a strong developer. They were fluent wherever the knowledge has been documented and improvising wherever it hasn't — and the gaps were never in Postgres or React or anything a degree covers. They fell entirely in one thin band: the operating layer between "I can drive this tool" and "this tool makes me better at my job." It is too basic for anyone who has it to think it worth writing down, and too new for anyone to have inherited it.
So I am writing it down.
One idea turned out to sit under all of it, and it arrived in the meeting as a throwaway about memory: if there is no proper memory support, day one is exactly as awkward as day ten.
That is true, and it is not really about memory. The model supplies intelligence; it does not supply continuity. It forgets the middle of its own context inside a single session, starts from zero when the next one opens, re-derives the same procedure every time it comes back to it, and is exactly as impressive on day two hundred as on day one — which sounds like praise and is not. The intelligence is real. The continuity is not, until you build it.
Most of what follows is the same move made at a widening scope. Within a session, continuity is managing your context, so you don't lose the thread by mid-afternoon. From one session to the next it is memory — and it keeps paying out for as long as the file survives, which can be months. For the procedure you run only now and then, it is a runbook. Across a whole project it is those artefacts together, the memory and the runbooks a fresh checkout inherits. And across a team it is the same again, one scope wider: a runbook or a memory file is how someone who was never in the room inherits what you worked out, instead of interrupting you to ask.
Three of the sections sit off that line, and it is worth saying which. Skills are not a scope at all — they earn their keep by giving the model the one thing it is weak at, its own process. The cost model is the constraint that makes the whole ladder urgent: context is finite and it degrades, so the room you free is the room you get to think in. And the habits at the end are the part that never accumulates in the tool at all — the work that stays yours. None of it is automatic, and all of it is cheap to skip, which is why almost everyone does.
Context is the thing you are actually managing
If I could install one habit in a developer working with agents, it would be this: at any moment in the day, be able to stop and enumerate what is in your context and roughly what it costs. Not precisely. Roughly.
Almost nobody can do this, including people who have been doing this work for a year. So the exercise is: guess first, then check. Every fifteen minutes, stop, say the number out loud, then look at the real one. A day of that is enough to build the intuition, and the intuition is the thing — you are not trying to become a token accountant, you are trying to stop being surprised. It is also the first thing here that accumulates: the calibration is yours, and it survives every compaction.
Here is what the intuition buys you. Long contexts do not fail at the advertised limit; they degrade well before it, and unevenly. Chroma's 2025 work on what they call context rot tested eighteen frontier models and found accuracy dropping non-uniformly as input grew — a model with a 200k window showing real degradation around 50k. The older lost-in-the-middle result has the same shape: retrieval accuracy is U-shaped by position, best at the beginning and the end, dropping by more than 30% for material buried in the middle. And the counter-intuitive part, which kills the obvious defence: on Chroma's retrieval task, coherent, well-structured input degraded performance more than shuffled input did. "But it's all relevant" is not an argument.
The way I explain the felt version is that the model remembers what happened two hundred thousand tokens ago roughly the way you remember being four years old. You remember which kindergarten you went to and where you lived. You do not remember how many chairs were in the room. Gist survives; detail does not — which is worse than forgetting, because an agent that has lost the detail does not report a gap. It confidently supplies a plausible number of chairs.
I owe a correction here, because I said something in that meeting that is wrong. I said that with 400k in context, a message saying "hi" costs you 400k on the way in and 400k on the way out. The input half is roughly right — the whole conversation is resent every turn. The output half is not: output tokens are what the model generates, which for "hi" is a handful. And the input half needs its own asterisk, which is caching: a warm prefix bills at about a tenth of fresh input, so 400k of cached context costs you something like 40k of fresh input on every turn — including the turn where you type "hi." That is the honest version of the number I got wrong. It is smaller and it is still absurd.
Which gives you the answer to when do I compact. Not a threshold alarm. You compact when your mental inventory of the context stops matching what the model is acting on — when it starts confidently misremembering a decision you can see was made. If you have been running the fifteen-minute exercise, you will notice this. If you have not, you will instead spend forty minutes arguing with an agent about a file it can no longer really see.
And one rule falls straight out of the U-shape: placement is a lever, not just size. The middle of a long context is where things go to be forgotten. Anything that must survive — the constraint, the invariant, the thing that will ruin your afternoon if it is dropped — goes at the top or gets restated at the bottom, not buried at turn forty of sixty.
Memory is what makes day ten different
Compaction is lossy, and obviously so: you cannot fit into ten thousand tokens what originally sat in four hundred thousand. Which means the operative question is not how do I compact well. It is: what must survive, and where do I put it so that it does?
That is what memory is for. Every serious assistant now has some form of it, and it is not exotic — it is a markdown file that gets loaded every time. What goes in it is unglamorous: this person prefers documents in their own language. Node 24. That one library needs an extra install step. The build breaks in this specific way if you skip that specific flag.
You are not training anything — you are teaching, the way you teach a new colleague, and memory is the channel. If you want a diagnostic: two weeks into a project, are you still seeing the same mistakes? If yes, memory is not being used properly. That is a good first test, and unforgiving in a useful way.
The strongest evidence I have for this is a rule of my own. Several repositories I work in are public, and nothing may be pushed to them without review. So there is a line in capitals at the top of the memory file saying never push to GitHub. I wrote it about four months ago. It has not been violated once — and when I say "push all the repositories," the thing understands that it may only push to our internal host.
The developer's immediate response to that story was the right one: can't that conflict with something? Yes. It can, and it is a little frightening to depend on. But notice what makes it hold: the context stays under about 300k, which is what keeps a line written at the top still reading as important. Let the context run toward a million and the things in my memory files start getting ignored — the soft ones, at least. The shouted one has held, which is not a guarantee; it is a sample size of four months. Your safety rules and your context budget are the same problem — and the rule lives at the top of the file, not the middle of it, for the reason the previous section gave.
Two failure modes to watch. The first is bloat: memory swells with near-duplicates — "always run the linter," then twenty lines later "the linter must always be run," then twenty lines after that "be sure to run standard linting operations" — until it contains repetitions and eventually contradictions. Memory needs compacting too. The second is sentiment about your own tooling. I built a whole system for capturing this kind of learning; over six months the platform absorbed most of what it did, and its usefulness dropped accordingly. Assume that will happen to your workaround, and retire it without ceremony when it does.
Skills: mostly not what you think
A skill is a markdown file. That is nearly the entire answer. It has a small metadata block — a couple of fields, including a short description that stays loaded — and then a body that gets pulled in when it is relevant.
The question worth asking is the second one: how can a skill exist in one place and not another? Not because of the file — the file is fine. The harness has to know it exists. Claude Code, or Codex, or whatever program you are actually working through has to be told to look there, for this project, on this machine. Hence per-project and per-machine skill collections, and formats that are only mostly compatible across tools. When a skill "doesn't work," the diagnosis is almost always loading, not authoring.
Now the part that gets me into arguments. For ordinary programming work — the spec exists, you need to write some Go — skills are needed approximately never. A "how to work with Go" skill is not neutral; it is worse than nothing. It occupies context, it biases the model toward whatever its author thought good Go looked like eighteen months ago, and the model already knows Go better than the skill does.
This is the first place the thesis cuts both ways. The point is not to accumulate artefacts; it is to accumulate the ones the model cannot regenerate. Where skills earn their keep is process. Models are mediocre at organising their own development process; told "go do it," they will do something confidently and not the thing you needed. Planning, brainstorming, debugging discipline, test-first — that is where a skill pays for itself, because you are supplying the one thing the model lacks rather than duplicating the thing it has. The narrow exception on the technical side is a specific external system's API, which must be in context or the model will simply invent it.
And then: open the file and change it. Do not like how a skill behaves? Edit it. Do not understand what a skill does? Read it, be surprised, then edit it. My own end-of-day report skill is six months old and works mostly off commit messages, because nobody has ever told me they wanted anything else. When the developer heard that, they drew the correct conclusion immediately — then I need to write more detailed commits. Both halves of that loop are available to you. Write better commits, or change the tool that reads them. These are files, not features.
Runbooks: the cheapest thing you are not doing
This idea has nothing to do with AI. It comes from support engineering and SRE, and the motivating scenario is mundane: you need to back up a database. You do not know where it lives, what it is called, what the command is, or how to get onto the machine. So you interrupt a developer to ask the database name, then a sysadmin to ask where the keys are. And you will do it again next quarter.
A runbook ends that. The standard is higher than "instructions": it is copy, paste, and it works — and each step states what you should see when it worked. That last part is the one everyone skips, and it is the only part that helps when you are tired. The word gets used loosely, and playbook gets used for roughly the same thing, including by me. Define whichever you mean and move on.
The rule is simple: the second, third, fifth time you do something, write the runbook. In this meeting, the developer told me they had needed to update a machine, found no documented procedure anywhere, and had to pull a colleague away to learn it — and then said something I have been thinking about since:
I have never been on a project that had a runbook for how to update things.
Neither had I, for most of my career. We agreed that this is bad. It is worse now than five years ago, because the knowledge that used to live in the team's collective memory — and walked out the door whenever a person did — now has a second consumer that reads documents very well and asks no questions. Models write runbooks competently, and — the underrated part — they can execute an under-specified one, working out for themselves where to go and what to call. The copy-paste bar was set for tired humans. For a model, it drops to the facts that cannot be inferred: the hostname, the database name, where the credentials live, which of the three plausible commands is the real one.
Write to the human bar anyway. The model will forgive you the steps you skip; the person doing this at 2am will not.
One honest counterpoint. Sometimes the correct response to a painful recurring procedure is not a better runbook but deleting the procedure. I recently loosened a protocol specifically so that adding a new device type no longer requires republishing a package and repinning every consumer — the procedure was fine, the need for it was the defect. So: write the runbook the second time. On the fifth, ask whether the procedure should exist at all. Either way something is left behind that was not there before. Day ten should not look like day one.
The cost model you are actually operating
Cost regressions are silent. Nothing falls over; the bill is just quietly wrong, and since providers change pricing almost monthly you have nothing stable to compare against. So the artefact that matters here is not a saving — it is a baseline: what a normal day costs, written down, so that an abnormal one is visible. Two things move it, and neither is the model you picked.
The first is caching, which you have already met: it is why 400k of cached context bills like 40k of fresh input on every turn. A warm read costs about a tenth of a cold one — an order of magnitude — but two mechanics matter more than that discount. The cache is per model, so hopping to another model and back pays the write again; and it is a prefix cache, so changing anything early (the system prompt, the tool definitions) invalidates everything after it. The rule people take from this is subtly wrong: not never switch models, but don't churn the front of your context — and a workflow that churns it has no stable baseline to write down anyway.
The second is language — and here I am the folklore. I have told people that working in Russian costs five times what English does — "апельсин" versus "orange" — and always hedged that I was exaggerating. I was: апельсин is four tokens, not the eight to ten I claimed. Isolated words do run 3–7×, and the technical nouns fragment worst (развёртывание is five to seven). So I expected a dense paragraph of deployment Russian to add all of that up, and measured one against its English to see. It does not add up. On a current large-vocabulary tokenizer the paragraph comes out 1.5× — no worse than ordinary prose, because the per-word cost washes out among the spaces and common words a modern tokenizer handles fine. On a previous-generation tokenizer the same paragraph is 2.4×. That is where the folklore of "two or three times" comes from — and where it still lives, if you are on an older or smaller model.
Here is what I had backwards when I first told this as a cost story. It is a productivity lever, not thrift. Caching is the bigger discount, but it is a discount on the bill; this is a discount on context, and context — which rots long before the window fills — is the constraint that actually binds. Every token spent on a language the tokenizer handles badly is a token not spent on the problem. Shave a third off on a frontier model, or better than half on the small models many people actually run, and you have not saved money — you have fit that much more work into the same context before it degrades. On those smaller models it is more than double the work in a single pass: not a rounding error, a different job. The under-spend is the same mistake inverted: a metered budget sitting ninety percent unused on Friday is not thrift, it is capacity that expired. Spend it — on the language that fits more in, and on the work you were afraid to attempt.
The quality case for English survives all of this on its own terms — these models are simply better in it — but it has to be made as a quality argument, not smuggled in as a token count.
So: write down what a normal day costs you, this week. The next pricing change will remove your ability to reconstruct it.
The habits at the human edge
Three practices that are not about the tools at all, and which decide more than the tools do.
Estimate, then do the retro. Estimation is a skill, it is learnable, and it is one you can straightforwardly fail at — a failed estimate is as much a developer failure as a failed deploy. How do you estimate something you have never done? The same way you estimate how long it takes to read a detective novel from the title alone. You know roughly what the category costs — modern detective novels run about 250 pages and are not dense, so: one evening — and then you adjust for the one alarming noun in the title. That is reference-class forecasting, and it is the documented answer to the planning fallacy, which is otherwise robust across cultures, personalities and experience, and which knowing about does not fix. Only outside data fixes it. Where you have no class at all, you are not forecasting, you are anchoring — so anchor deliberately: a colleague's "couple of hours to get your head round it" is a floor, not an estimate.
But the estimate is not the practice. The retro is, and it is the step almost everyone skips, which is why most developers do not improve at this. I know people who estimated with a factor-of-four error a decade ago and estimate with a factor-of-four error today, because the moment never arrives where they ask: I said four, it took sixteen, why? The answer is usually not "I was optimistic about the code." It is rework after someone reinterpreted the task, and deployment time, and the twenty other things that never make it into the estimate because they are not the interesting part. You learn those once, by looking.
Reviewing an agent's work is a different job from reviewing a colleague's. When I put agent-authored merge requests in front of a human, what I want back is not merge or reject. I want the critique: what is wrong with this, and what would you have wanted to see instead — feedback aimed at whoever owns the agent, not at the diff. The diff is the symptom. And the way to calibrate is to spend duplicate effort deliberately: run the agent on a task a human is also doing, then compare. The comparison tells you whether the failure was the agent or the spec — the only thing that tells you where to spend the next hour.
Keep the thing they cannot do, and keep it in working order. My original plan, when I put agents into production back in February, was that they would handle ninety percent of routine incoming tasks by themselves. I spent a long time trying to persuade them to do it. It did not work. What stays ours is contact with physical reality: the code gets written, and a human still has to run it on the actual hardware, on the actual operating system, and look. But that advantage is only real if you can exercise it on demand, which makes the test bench a capital asset. Book it, put it in a known state, learn to power-cycle it. Then it is an advantage.
Back in the room
What I actually asked for at the end of that meeting was not "read this guide." It was: write down, for yourself, what you know and what you have decided not to know — and then bring it to me so we can argue about it, because I have a better picture than you do of which of those things will still matter next year. Which competences you should hold and which you can rent is a genuinely separate question from everything above, and I have argued it at length in Renting Competence — the one-line version being: don't LLM yourself out of the job. I will not re-litigate it here.
But I will note the shape the meeting had, because it embarrasses the usual seniority story. I opened by asking them what I was missing — they have been using tools I have not touched, and in a landscape that turns over monthly, being senior confers no coverage. You can see it better than I can right now, and so will whoever reads this next.
None of it compounds by default. Every practice above is a small decision to make the work accumulate instead of evaporate, and skipping them is free, and permanent: the tool stays impressive forever, you stay exactly where you started, and whoever comes after you inherits nothing, because nothing was left where they could find it.
Day ten should not look like day one.
Sources
- Chroma (2025), Context Rot: How Increasing Input Tokens Impacts LLM Performance — the 18-model study behind the degradation-before-the-limit claim. trychroma.com
- Liu et al. (2023), Lost in the Middle: How Language Models Use Long Contexts — the U-shaped position curve. arxiv.org/abs/2307.03172
- Anthropic, Prompt caching — cache reads at 0.1× of input (the order-of-magnitude discount); writes carry a short/long-TTL premium. platform.claude.com
- Kahneman & Tversky (1979) on the planning fallacy; Flyvbjerg (2006), From Nobel Prize to Project Management: Getting Risks Right, for reference-class forecasting as its operational correction. arxiv.org/pdf/1302.3642
- Token counts measured locally with
tiktoken0.11.0 (cl100k_base,o200k_base).
No comments yet