Never rewrite the system prompt

Problem: Agents re-render the date, branch and rules into the system prompt every turn, which breaks the provider's prompt cache.
Idea: Freeze the system prompt for the conversation, and append a short update at the next turn when something changes.

Model providers cache the start of a request. If the prefix is byte-for-byte the same as last turn, you pay a fraction of the price and wait less. Many agents break that cache every turn without noticing, because the date, the git branch or the active rules are re-rendered into the system prompt.

Try it: what the cache can reuse

green is served from the provider's cache, red is paid in full

If the system prompt changes every turn, nothing before the change can be reused, so every request is paid in full. Freeze the prompt and append small updates instead, and each turn only pays for what's new.

The model

Typed context sourcesThe system context is built from Context Sources, each with a stable namespaced key, a JSON codec, a loader and pure renderers for the baseline, an update and, if needed, a removal.
A frozen baseline per epochAt the start of a Context Epoch the full context is rendered once and becomes "the immutable provider-cache baseline", until a completed compaction, a session move or an incompatible change.
A snapshot of what the model sawA hidden JSON snapshot stores the last value admitted for each source, so the runtime can tell exactly what changed.
Appended updatesWhen a source changes, a Mid-Conversation System Message with its newly effective state is appended at the next safe boundary. The baseline is never edited.

The rules that make it correct

Lazy, not pushed

Changes are "sampled and admitted lazily at a Safe Provider-Turn Boundary, never pushed asynchronously when their source changes."

Deterministic

Producers run concurrently but combine in stable key order; duplicate keys fail. Changes at one boundary combine into a single message.

Stale beats missing

If a source can't be read right now, the previous value stays in effect and nothing is emitted. A source that's really gone can emit removal text.

Atomic and replayable

The snapshot advances in the same step as the durable update message, which stores the exact text the model was sent.

Where the epoch ends

Compaction starts a new epoch with a freshly rendered baseline and snapshot; earlier update messages stay as audit history but leave the model's view. If the initial context can't be loaded, the first turn is blocked rather than persisting an incomplete baseline, and the prompt stays in the inbox to retry.

Steal this

Render the system prompt once per epoch and never edit it. Track what the model last saw for each piece of context, append changes at turn boundaries in a stable order, and let compaction be the only thing that starts a new baseline.

Use it in your code

A prompt for your coding agent

Run it from the root of your repo. It checks whether the idea fits before it changes anything.

Make this app's LLM requests cache-friendly by never rewriting the start of the conversation.

The idea: providers cache the beginning of a request. Render the system prompt once and keep it byte-for-byte stable. When context changes (time, git branch, files, settings), append a short update message at the next turn instead of editing the system prompt.

1. Find where system prompts and context are built for model calls, and list every value that can change between turns: timestamps, dates, environment details, open files, settings, tool lists.
2. Tell me which of these currently break the provider's prompt cache on every turn, and estimate the cost if you can (tokens re-sent per turn).
3. Propose a structure: a stable baseline rendered once per conversation (and again after compaction), a snapshot of what the model last saw for each changing value, and an appended "context update" message only when a value changes, in a stable order.
4. Implement it for the noisiest value first, with a test that two turns without changes produce an identical prefix.

If the provider reports cache hits, show me the before and after.

Read the source

Where this comes from

OpenCode (anomalyco/opencode), the open-source coding agent (these lessons are from its upcoming v2). The files behind this lesson: