A Lossless Diet for an AI Agent's Bloated Instruction File

Most AI coding agents load a project instruction file into every session. It starts as a page of conventions and slowly becomes the team's junk drawer: deployment runbooks, incident postmortems, schema migration notes, frontend quirks. Ours reached 253 lines and 67KB, roughly 33,000 tokens, injected before the agent had read a single line of code. This post describes the restructuring that cut it to a 10KB routing hub, the safety rules that had to stay behind, and the verification that proved the move lost nothing. The constraint that shaped everything: no content could be deleted, only moved or merged.

Why instruction files bloat: a rule without a destination

The file did not grow because anyone was careless. It grew because of a rule that sounded responsible: "whenever a policy or setting changes, update the instruction file immediately." The rule had no routing. Every project added its operational detail to the one file that was guaranteed to be read, because that was the only place a future agent was guaranteed to look.

That is the structural cause worth recognizing in your own setup. A single always-loaded file with an append-only culture grows monotonically. Nobody is ever in a position to remove a warning, because every warning earned its place through some past incident. After a year we had table cells with ten paragraphs of history in them, and the agent paid the token cost on every session, including sessions that never touched the subsystems those paragraphs described.

The waste is not only money. A 33,000 token preamble dilutes attention. Models demonstrably lose instructions buried in the middle of long context. The warnings we most needed the agent to obey were competing with paragraphs about a crawler quirk from months ago.

Split by topic, but keep the stop gates

The fix is the obvious one: move detail into topic documents that are read on demand, and keep the hub small. We ended up with eight operations documents (deploy, cloud resources, admin web, licensing, speech pipeline, data collection, crawling, nightly batch) plus a migrations README that lives next to the migration files themselves. Each document owns its domain fully. The hub keeps one line per topic and a pointer.

Two design decisions mattered more than the file layout.

First, the index cannot be a polite list of links. An agent under time pressure will skim a list and start coding. We wrote it as a conditional routing table: "modifying licensing or seat logic: you MUST read docs/ops/license.md first." The difference sounds cosmetic. In practice, an instruction phrased as a precondition gets obeyed; a link phrased as a reference gets skipped. We also added one line telling the agent to grep the docs directory for a keyword instead of reading files whole when it is unsure where something lives.

Second, and this is the part that prevents accidents: some warnings must not move. If "run pending schema migrations before deploying" lives only in a document the agent might not open, then one day the agent will deploy without reading it, and the incident that warning encodes will happen again. We kept seven of these in the hub under a heading called Stop Gates, each one a summarized invariant with its details delegated:

• Check migration state before any deploy; if unclear, do not deploy. Roll back by reverting the image first; only then consider reverting the schema.
• Only use additive flags when changing managed runtime environment variables; the replace-everything variants silently wipe unrelated config.
• Never write user speech transcripts, translations, or secret values into logs, documents, or commits.
• The database IP allowlist flag overwrites the whole list; read the current list first and patch the union.
• Retention and consent rules for collected audio are invariants; read the data document before touching them.
• Destructive or overwrite-style commands against production stores require target, blast radius, and rollback confirmation first.
• Never bypass, delete, or weaken paid endpoint auth, rate limiting, or fail-closed trial logic.

The selection rule we used: a warning stays in the hub if ignoring it causes an outage, data loss, a compliance breach, or a customer-visible failure, and moves out if ignoring it merely wastes an afternoon. Cost of ignorance, not frequency of use.

One more line went at the very top of the hub, above everything: when documentation and code disagree, the code is the truth; update the document, never "fix" working code to match a stale document. A split documentation system fails worst when an agent finds a contradiction and resolves it in the wrong direction.

Proving the move lost nothing

"We moved everything, trust us" is not verification. Our first plan was to spot-check a handful of distinctive tokens with grep. A reviewer pointed out three failure modes that sampling cannot catch: a token can survive while the sentence around it is gutted; the union of old and new files passes even if text never left the old file; and text can land in the wrong document. Sampling proves the samples, nothing else.

What we ran instead, all mechanical:

First, extract every inline code span from a snapshot of the original file taken before editing. That means every backtick-quoted command, flag, path, and identifier. We got 583 unique spans. Each one must appear somewhere in the union of the new documents, compared as fixed strings after whitespace normalization, not as regular expressions, because spans like $1::text[] are full of regex metacharacters.

Second, extract every sentence containing a warning marker (the words for forbidden, required, caution, and the warning emoji our docs use). We got 61 sentences. Because the migration was verbatim moves rather than rewrites, each sentence had to exist somewhere, character for character after normalization. This is the trick that makes the check strong: a moved sentence carries its conditions, its severity, and its reasons with it. Only three sentences failed the check, and each had a documented excuse: one was merged with a duplicate, one was split across table rows, one had an internal cross-reference deliberately rewritten. Anything without an excuse would have been a stop-ship.

Third, byte accounting as an anomaly detector, not a gate. New files summed to 129 percent of the original (headers, pointers, and the new Stop Gates section add text). If that number had come in at 60 percent we would have gone hunting. We deliberately did not make it a pass-fail threshold, because a legitimate dedup pass can shrink the total while a bad move can pass the threshold with the wrong text intact.

The span check earned its keep immediately. One sentence, a note that the web server subcommand is distinct from a similarly named realtime subcommand, had fallen through a gap in the migration map. No human reading 67KB twice would have caught one missing clause. A fixed-string comparison over 583 spans caught it in seconds and we restored it before shipping.

Smoke-test the routing with a fresh agent

Text verification proves preservation. It does not prove the new layout works. For that we gave a fresh agent session only the new 10KB hub and asked three questions: what do you read before modifying seat licensing logic, how do you deploy the admin web service, and please wipe the cache in production.

The first two answers routed to the correct documents and cited the migration gate. The third is the one worth designing for: the agent refused to run anything, cited the destructive-work stop gate, asked which environment and which key prefix, and proposed a scoped deletion instead of a full flush. That refusal was the whole point of keeping stop gates in the always-loaded file. If your smoke test cannot make the agent refuse something, the test is too soft.

Keep it small: fix the rule, not just the file

The last change was the one that prevents a rerun of the whole exercise in six months. The original growth rule ("always update the instruction file") was replaced with a routing rule: standing rules, coordinates, and stop gates go in the hub with a hard cap of 120 lines and 10KB; operational detail, flags, and incident history go in the topic document that owns the domain; decision rationale goes in the design log; and code-local traps go in code comments next to the code they guard. When the hub exceeds the cap, the required response is to split content out, not to negotiate the cap.

A few closing numbers for scale. The hub went from 67KB to under 10KB, a session-context saving of roughly 28,000 tokens on every single agent run. The verification suite is about 40 lines of Python and shell. The whole migration, including three rounds of review and the verification tooling, was a day of work. If your agent's instruction file has crossed a few thousand tokens and keeps growing, the diet is cheap, the token savings compound daily, and with span-level verification you do not have to choose between a small file and a safe one.