Skip to content
BSWEN
Blog

How Prompt Caching Works in Claude Code: Cache Prefix Explained

Cowrie
Cowrie
Dev @ Bswen

Problem

When I resumed a Claude Code session after a break, I noticed my token costs spiked. The cache hit ratio dropped from 96% to under 30%, even though I was continuing the same conversation.

Here’s what I saw in the usage stats:

// Fresh session - good caching
{
  "usage": {
    "cache_read_input_tokens": 57484,
    "cache_creation_input_tokens": 611,
    "input_tokens": 1500
  }
}
// Ratio: 96% cached


// After resume - broken caching
{
  "usage": {
    "cache_read_input_tokens": 2100,
    "cache_creation_input_tokens": 55895,
    "input_tokens": 1500
  }
}
// Ratio: 3.6% cached - almost all re-processed!

Why does resuming a session destroy the cache? I dug into how Claude Code’s prompt caching actually works.

Environment

  • Claude Code CLI
  • Claude API with prompt caching enabled
  • Extended conversation sessions

What Is Prompt Caching?

Anthropic caches the “prefix” of your conversation to reduce costs. When you send a new message, the cached portion is read cheaply, while only new content gets full processing.

There are two token types:

  • cache_read_input_tokens: Cheap - reading from existing cache
  • cache_creation_input_tokens: Expensive - writing new content to cache

The goal is simple: maximize cache_read, minimize cache_creation.

But there’s a catch. The cache relies on a stable “prefix” - the initial messages in your conversation array. If that prefix shifts, the entire cache breaks.

How Cache Prefix Works

The cache prefix includes:

  1. System prompts and reminders
  2. Tool definitions
  3. Previous conversation messages

Anthropic computes a “billing hash” from your first user message. This hash identifies the cache entry.

Here’s a fresh session structure:

messages[0]: system_reminder (cached)
messages[1]: tool_definitions (cached)
messages[2]: user_message (billing_hash anchor)
messages[3-N]: conversation

Everything from index 0 up to the cache_control breakpoint gets cached. New messages at the end only incur input_tokens, not cache_creation.

What Breaks the Prefix

Three things can shift the prefix and break caching:

1. System reminders moving

If system_reminder moves from messages[0] to messages[N], the entire array shifts.

2. Billing hash changing

The first user message determines the billing hash. If that message’s content changes or moves to a different index, the hash changes.

3. cache_control breakpoint shifting

The breakpoint marks where caching stops. If messages are inserted before this point, the cached content changes.

Why Session Resume Breaks It

I traced through what happens when Claude Code resumes a session.

The db8 function (a session handler) strips deferred_tools_delta records. These records track which tools were already announced. When stripped, Claude Code re-announces all tools on resume.

This inserts new messages at the start of the array:

Resumed Session (broken):
messages[0]: NEW tool_announcements (breaks anchor!)
messages[1]: system_reminder (shifted - no longer cached)
messages[2]: tool_definitions (shifted)
messages[3]: user_message (shifted - hash changes!)
messages[4-N+1]: conversation (shifted)

Every message shifts by one position. The billing hash anchor moves. The cache_control breakpoint moves. The entire cache becomes invalid.

Cache Ratio Math

I wrote a quick script to monitor my cache ratio:

Terminal window
# Monitor your cache ratio over time
watch -n 5 'tail -20 ~/.claude/projects/*/*.jsonl | python3 -c "
import sys, json
for line in sys.stdin:
    try: d = json.loads(line.strip())
    except: continue
    u = d.get(\"usage\") or d.get(\"message\",{}).get(\"usage\",{})
    cr = u.get(\"cache_read_input_tokens\", 0)
    cc = u.get(\"cache_creation_input_tokens\", 0)
    if cr or cc:
        print(f\"Read: {cr:,}  Created: {cc:,}  Ratio: {cr/(cr+cc)*100:.0f}%\")"
'

The ratio formula:

ratio = cache_read / (cache_read + cache_creation + input_tokens)

Ratios:

  • 90%+: Good - mostly cached
  • 50-90%: Acceptable - some re-processing
  • <30%: Bad - cache prefix likely broken

Visual Comparison

FRESH SESSION:
┌─────────────────────────────────────────┐
│ messages[0]: system_reminder            │ ← CACHED
│ messages[1]: tool_definitions            │ ← CACHED
│ messages[2]: user "start conversation"   │ ← billing hash
│ messages[3]: assistant response         │
│ messages[4]: user "continue..."          │ ← NEW (cheap)
└─────────────────────────────────────────┘


RESUMED SESSION:
┌─────────────────────────────────────────┐
│ messages[0]: tool_announcements (NEW!)   │ ← BREAKS CACHE
│ messages[1]: system_reminder (shifted)   │ ← NOT cached
│ messages[2]: tool_definitions (shifted)  │ ← NOT cached
│ messages[3]: user "start..." (shifted)   │ ← hash changes!
│ messages[4]: assistant response          │
│ messages[5]: user "continue..."          │
└─────────────────────────────────────────┘

The shift from tool re-announcement cascades through everything.

The Root Cause

The issue is not the cache mechanism itself. It’s how Claude Code manages session state:

  1. Session storage strips tool delta records for cleanliness
  2. Resume reconstructs the session without those records
  3. Claude Code re-announces tools to ensure they’re available
  4. Re-announcement inserts messages at array start
  5. Insertion shifts all indices
  6. Shifted indices break the cache prefix stability

It’s a trade-off between session portability and cache efficiency.

Summary

In this post, I explained how Claude Code’s prompt caching relies on message prefix stability. The key points:

  • Cache prefix includes system prompts, tools, and conversation history
  • Billing hash anchors to the first user message
  • Session resume re-announces tools, shifting all message indices
  • Shifted indices invalidate the entire cache prefix
  • Monitor your cache ratio to detect prefix breaks

Understanding this helps diagnose token usage spikes and optimize workflow. For long sessions, consider starting fresh rather than resuming if cache efficiency matters.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!

Comments

Tags: