Anthropic's prompt caching feature lets the model reuse cached KV state for repeated context. For a tutor product with a long, static system prompt and a per-problem context that persists across many turns, this is a large cost lever.
We cache every system prompt in the codebase with cache_control: ephemeral. Tutor, hint, reveal, grader, classifier, leak guard, recall rephrase, normal assistant — all cached. Cache TTL is around five minutes, which is longer than most single-session engagement.
We additionally cache the per-problem context — the problem statement plus the student's confidence tag — as a second block. This means multiple turns on the same problem (attempt → tutor reply → student re-attempt → tutor reply) share the same cache.
The economics are favorable. On active sessions we see cache hit rates over 80% on system prompts and roughly 60% on per-problem context. Total inference cost is down about 45% versus an uncached baseline.
The engineering cost was two lines per prompt call. This was one of the highest-leverage engineering weeks in the project.
Written by Brainback Engineering. Published Jul 16, 2026. Filed under Method.