Why Claude Code Burns Through Context Window So Fast (And How to Fix It)

Mar 20, 2026

The Problem

When I use Claude Code on a large codebase, I notice something frustrating. It loads entire files just to find a single function definition. Context window fills up fast. Tokens get wasted on irrelevant code.

I was working on a TypeScript monorepo with 500+ files. Claude Code would read whole files to understand symbol relationships. By the time it gathered enough context, half the context window was gone.

The core issue: LLM code assistants don’t know where symbols live. They search through files, read everything, and hope to find what they need.

What’s Really Happening

Let me trace through what happens when I ask Claude Code to “find the implementation of getUserById”:

User: "Find the implementation of getUserById"

Claude Code:
1. Searches files for "getUserById" text match
2. Reads entire files that contain matches
3. Parses code mentally (expensive tokens)
4. Finds definition buried in a 500-line file
5. Returns the function

Result: 10,000+ tokens used for what should take 200

The inefficiency compounds. Every symbol lookup repeats this process. The context window becomes a dumping ground for entire files when we only need specific symbols.

I ran into this constantly. A simple refactoring task required 5-6 context-window resets because each lookup consumed thousands of tokens.

The Solution: Symbol Indexing

The fix is straightforward in concept: build a map of where every symbol lives, then retrieve only the symbols we need.

Without Index:
  Query → Read entire files → Parse → Extract symbol → Waste tokens

With Index:
  Query → Lookup symbol location → Read only that symbol → Save tokens

A Reddit post caught my attention. Someone pair-programmed ~22K lines of C with Claude Opus to build a symbol indexing tool. They indexed:

Redis codebase: 727 files, 45K symbols
Linux kernel: 65K files, 7.4M symbols

The key insight: deterministic parsing (not LSP-based) can handle massive scale while enabling precise symbol retrieval.

Three Approaches to Symbol Indexing

Approach 1: Deterministic Parsing (tree-sitter, ctags)

tree-sitter creates concrete syntax trees from source code. It’s incremental, fast, and language-agnostic.

ctags generates tag files mapping symbols to file locations. Lightweight and widely adopted.

tree-sitter output (conceptual):

File: src/user_service.ts
Symbols found:
├── class UserService (line 12-45)
│   ├── method getUserById (line 18-25)
│   ├── method createUser (line 27-35)
│   └── method deleteUser (line 37-44)
├── interface User (line 5-10)
└── function validateEmail (line 48-52)

Index entry:
{
  "name": "getUserById",
  "type": "method",
  "file": "src/user_service.ts",
  "line_start": 18,
  "line_end": 25,
  "parent": "UserService"
}

Strengths:

Fast indexing (milliseconds to seconds)
No external dependencies
Works offline
Scales to millions of symbols

Limitations:

No semantic understanding (types, cross-file references)
Limited language intelligence

Approach 2: Semantic Analysis (LSP)

Language Server Protocol provides IDE-grade intelligence: go-to-definition, find-references, type checking.

LSP provides richer data:

Symbol: getUserById
├── Definition: src/user_service.ts:18
├── References:
│   ├── src/controllers/auth_controller.ts:45
│   ├── src/controllers/admin_controller.ts:23
│   └── tests/user_service.test.ts:112
├── Type signature: (id: string) => Promise&lt;User | null&gt;
└── Implementation details

Strengths:

Deep code understanding
Accurate reference resolution
Type-aware navigation

Limitations:

Requires language server per language
Higher memory and CPU usage
Slower initial indexing
May struggle with Linux kernel-scale codebases

Approach 3: Hybrid

Combine both approaches for best results:

Query → Symbol Index (tree-sitter) → Filter → LSP (if needed) → Context

Example flow:
1. tree-sitter finds "getUserById" in user_service.ts (fast)
2. LSP resolves type signature and references (semantic)
3. Return minimal context

Why Deterministic Parsing Won for Massive Scale

The Reddit implementation used pure C with deterministic parsing. No LSP servers. No semantic analysis. Just symbol extraction.

I thought about why this works:

1. Symbol retrieval doesn’t need semantic depth

Most lookups are simple: “Where is function X defined?” The answer is a file path and line numbers. You don’t need type inference for that.

2. Scale breaks assumptions

LSP servers assume reasonable codebase sizes. Linux kernel has 7.4M symbols across 65K files. That breaks LSP assumptions about memory and startup time.

3. Deterministic is reproducible

No “smart” heuristics that might fail unpredictably. You get the same index every time.

4. Minimal dependencies matter

Pure C means no runtime dependencies. Works in any environment. No Node.js version conflicts, no Python virtualenv issues.

When to Use Each Approach

Scenario	Recommended
Linux kernel scale (millions of symbols)	tree-sitter / ctags
TypeScript monorepo (type safety matters)	LSP (tsserver)
Mixed language project	Hybrid
Quick navigation, CI/CD pipelines	ctags
Accurate refactoring	LSP
Offline environments	tree-sitter / ctags
Real-time collaboration	LSP

Implementation Pattern

For LLM code assistants, the retrieval pattern looks like this:

# Conceptual: Symbol-aware context retrieval
class SymbolIndexRetriever:
    def __init__(self, index_path: str):
        self.index = load_symbol_index(index_path)

    def get_context(self, query: str, max_tokens: int) -> str:
        # 1. Extract symbol references from query
        symbols = extract_symbol_references(query)

        # 2. Look up symbols in index
        locations = [self.index.lookup(s) for s in symbols]

        # 3. Retrieve symbol definitions within token budget
        context = ""
        for loc in prioritize(locations):
            symbol_code = read_symbol_at(loc)
            if token_count(context + symbol_code) <= max_tokens:
                context += symbol_code
            else:
                break

        return context

The key: retrieve only relevant symbols, not entire files.

Integrating with MCP Servers

Model Context Protocol (MCP) servers can expose symbol indexing to LLM assistants:

{
  "mcpServers": {
    "symbol-index": {
      "command": "symbol-index-server",
      "args": ["--index", "/path/to/codebase.idx"],
      "tools": [
        "lookup_symbol",
        "find_references",
        "list_symbols_in_file"
      ]
    }
  }
}

This pattern lets Claude Code query symbol locations directly without loading entire files.

Performance Comparison

Approach	Index Size (100K lines)	Cold Start	Warm Query
ctags	~500KB	1-10ms	~1ms
tree-sitter	2-5MB	5-50ms	1-5ms
LSP	5-10MB	100-1000ms	10-50ms

For LLM context efficiency, the difference between loading 500KB (ctags index) vs 50MB (entire codebase files) is dramatic.

What I Learned

After understanding the problem and solutions:

1. Context window efficiency matters more than I thought

Wasting tokens on irrelevant code means fewer turns, more resets, and higher costs.

2. Deterministic parsing scales surprisingly well

I assumed LSP was always superior. The Linux kernel example proved otherwise. Sometimes simpler is better.

3. Hybrid approaches offer the best balance

Use tree-sitter for fast lookups, fall back to LSP when semantic understanding matters.

4. MCP servers are the right abstraction

Exposing symbol indexing as an MCP server makes it accessible to any LLM assistant, not just one tool.

Summary

In this post, I explained why LLM code assistants waste context window space and how symbol indexing solves this problem. The key insight is that retrieving only relevant symbols (not entire files) dramatically improves context efficiency. Deterministic parsing with tree-sitter or ctags scales to millions of symbols while keeping retrieval fast. For most use cases, a hybrid approach that combines fast indexing with selective semantic analysis provides the best balance.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

👨‍💻 Reddit: Pair-programmed 22K lines of C with Claude Opus to fix Claude Code's inefficiency
👨‍💻 tree-sitter Documentation
👨‍💻 Universal Ctags
👨‍💻 Language Server Protocol Specification
👨‍💻 Serena - LSP-based MCP Server

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!