How to Reduce Token Usage When Claude Code Reads Large Files

Mar 20, 2026

Problem

I was working on a large C project with Claude Code when I watched 84,000 tokens vanish in a single request. All I wanted was to understand one function: initServer(). But my codebase had 22,000+ lines spread across multiple files.

Request: "Explain the initServer function"
Claude Code: [Reads entire main.c file: 22,000 lines]
Token cost: 84,193 tokens
My reaction: That's 84K tokens for ONE function?!

At this rate, I’d burn through my token budget before finishing a single debugging session. Something was fundamentally wrong with how Claude Code was handling my codebase.

What happened?

I assumed Claude Code would be smart enough to locate and extract just the function I needed. After all, it’s an AI coding assistant—it should know how to navigate code efficiently.

Here’s what I observed:

My codebase structure:
- main.c (22,000 lines)
- server.c (8,000 lines)
- utils.c (5,000 lines)
- config.c (3,000 lines)
Total: ~38,000 lines of C code

My request: "Find and explain initServer()"
Claude Code behavior: Read entire main.c file
Token consumption: 84,193 tokens for ONE function lookup

I tested this multiple times. Every time I asked about a specific function, Claude Code would read the entire containing file:

Test 1: "What does handleRequest() do?"
Result: Read all of server.c (8,000 lines) = 31,000 tokens

Test 2: "Show me parseConfig()"
Result: Read all of config.c (3,000 lines) = 12,000 tokens

Test 3: "Find the utils helpers"
Result: Read all of utils.c (5,000 lines) = 19,000 tokens

The pattern was clear: Claude Code’s default behavior is to read entire files, even when you only need a small piece.

Why is this happening?

I dug into how Claude Code works internally. The issue stems from its file reading approach:

How Claude Code reads code:
1. User asks about function X
2. Claude Code needs to find X
3. Claude Code reads ENTIRE file containing X
4. Claude Code processes all tokens (including unrelated code)
5. Claude Code extracts and explains X

What should happen:
1. User asks about function X
2. System looks up X in an index
3. System returns ONLY the definition of X
4. Claude Code explains X

The problem is Claude Code lacks a symbol-level understanding of codebases. It treats files as atomic units, not collections of functions, classes, and variables.

Think of it like this:

Traditional approach (what Claude Code does):
User: "Get me chapter 5 from this book"
Action: Photocopy entire 500-page book
Result: User gets chapter 5, but you paid for 500 pages

Optimized approach (what I needed):
User: "Get me chapter 5 from this book"
Action: Use table of contents, photocopy pages 87-102
Result: User gets chapter 5, paid for 16 pages

For small projects, reading entire files is acceptable. For enterprise codebases with thousands of lines per file, it becomes prohibitively expensive.

The solution: MCP-based code indexing

I needed a way to retrieve only the symbols (functions, classes, definitions) I needed, not entire files. The solution was to build an MCP (Model Context Protocol) server that indexes codebases at the symbol level.

Architecture overview

Without MCP Server:
┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│ User Query  │────▶│ Claude Code  │────▶│ Read File   │
└─────────────┘     └──────────────┘     │ (84K tokens)│
                                         └─────────────┘

With MCP Server:
┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│ User Query  │────▶│ Claude Code  │────▶│ MCP Server  │
└─────────────┘     └──────────────┘     └──────┬──────┘
                                                │
                    ┌──────────────┐            │
                    │ Symbol Index │◀───────────┘
                    │  (SQLite)    │
                    └──────┬───────┘
                           │
                    ┌──────▼───────┐
                    │ Return only  │
                    │ initServer() │
                    │ (2.7K tokens)│
                    └──────────────┘

The MCP server pre-indexes the codebase, storing each symbol’s location and definition. When Claude Code needs a function, it queries the index instead of reading files.

Implementation steps

I built a code indexing MCP server with these components:

Step 1: Create the symbol database

import sqlite3
from tree_sitter import Parser, Language

class CodeIndexMCP:
    def __init__(self, db_path="code_index.db"):
        self.conn = sqlite3.connect(db_path)
        self.parser = Parser()
        self._create_tables()

    def _create_tables(self):
        """Create symbol storage table."""
        self.conn.execute("""
            CREATE TABLE IF NOT EXISTS symbols (
                name TEXT,
                file_path TEXT,
                start_line INTEGER,
                end_line INTEGER,
                definition TEXT,
                symbol_type TEXT,
                PRIMARY KEY (name, file_path)
            )
        """)
        self.conn.commit()

Step 2: Index the codebase

    def index_file(self, file_path: str):
        """Parse file and store all symbol definitions."""
        with open(file_path, 'r') as f:
            source = f.read()

        tree = self.parser.parse(bytes(source, "utf8"))

        for node in self._extract_symbols(tree.root_node):
            self.conn.execute(
                """INSERT OR REPLACE INTO symbols
                   VALUES (?, ?, ?, ?, ?, ?)""",
                (node.name, file_path, node.start_line,
                 node.end_line, node.definition, node.type)
            )
        self.conn.commit()

    def index_directory(self, root_path: str):
        """Index all source files in directory."""
        for file_path in self._find_source_files(root_path):
            self.index_file(file_path)

Step 3: Retrieve symbols on demand

    def get_symbol(self, name: str) -> dict | None:
        """Retrieve only the requested symbol definition."""
        cursor = self.conn.execute(
            """SELECT name, file_path, definition, symbol_type
               FROM symbols WHERE name = ?""",
            (name,)
        )
        result = cursor.fetchone()
        if result:
            return {
                "name": result[0],
                "file": result[1],
                "definition": result[2],
                "type": result[3]
            }
        return None

Step 4: Configure as MCP tool

{
  "mcpServers": {
    "code-index": {
      "command": "python",
      "args": ["/path/to/code_index_mcp.py"],
      "tools": [
        {
          "name": "get_symbol",
          "description": "Retrieve a specific function/class definition from indexed codebase",
          "inputSchema": {
            "type": "object",
            "properties": {
              "symbol_name": {
                "type": "string",
                "description": "Name of function or class to retrieve"
              },
              "file_hint": {
                "type": "string",
                "description": "Optional file path to narrow search"
              }
            },
            "required": ["symbol_name"]
          }
        }
      ]
    }
  }
}

Results after implementation

After deploying the MCP server, I tested the same initServer() lookup:

Before MCP Server:
Request: "Explain initServer()"
Method: Read entire main.c file
Tokens consumed: 84,193

After MCP Server:
Request: "Explain initServer()"
Method: Query symbol index, return definition only
Tokens consumed: 2,699

Reduction: 97% fewer tokens

The token savings were dramatic. For the same information, I went from 84K tokens to under 3K tokens.

Why this works

The efficiency gains come from three factors:

1. Pre-indexing vs. on-demand parsing

Traditional (Claude Code default):
Every query → Parse entire file → Extract relevant part
Time: O(n) where n = file size
Tokens: Proportional to file size

MCP Server:
One-time indexing → Store in database
Every query → Database lookup → Return definition
Time: O(log n) for indexed lookup
Tokens: Proportional to symbol size only

2. Symbol-level granularity

Instead of treating files as atomic units, the MCP server understands code structure:

File: main.c (22,000 lines)

Symbol index stores:
- initServer(): lines 150-180
- handleConnection(): lines 200-250
- parseConfig(): lines 300-340
- ...thousands more symbols

Query for initServer():
Returns: Only lines 150-180 (30 lines)
Not: All 22,000 lines

3. Reusable index

The index is built once and queried many times:

Initial setup:
1. Index entire codebase (one-time cost)
2. Store in SQLite database

Subsequent queries:
- Lookup initServer: 0.001 seconds
- Lookup handleConnection: 0.001 seconds
- Lookup parseConfig: 0.001 seconds

No re-parsing needed.

Common mistakes to avoid

When implementing code indexing, I made several mistakes:

Mistake 1: Over-indexing metadata

# WRONG: Storing too much metadata
def index_file(self, file_path):
    # Stores entire file as metadata
    self.conn.execute(
        "INSERT INTO symbols VALUES (?, ?, ?, ?)",
        (name, file_path, entire_file, all_comments)
    )
    # Result: Index becomes as large as original files

The index should store only what’s needed for retrieval, not duplicate the entire codebase.

Mistake 2: Ignoring index updates

# WRONG: Never updating the index
def get_symbol(self, name):
    # Returns stale definition after code changes
    return self.conn.execute("SELECT ...")

The index needs to track file modification times and re-index changed files.

Mistake 3: Not using existing tools

I initially tried to build everything from scratch. Better approaches exist:

Approach	Setup Cost	Token Savings	Maintenance
Custom MCP	High	97%	Medium
Serena	Low	High	Low
LSP-based	Medium	High	Low
ctags	Low	Medium	Low

For most projects, using an existing tool like Serena or integrating with LSP (Language Server Protocol) is more practical than building from scratch.

Alternative approaches

If building a custom MCP server is too complex, consider these alternatives:

Option 1: Serena

Serena is a code search tool that already implements efficient symbol lookup:

# Install Serena
pip install serena

# Index your codebase
serena index /path/to/project

# Query symbols
serena find initServer

Option 2: LSP Integration

Language Server Protocol already provides symbol lookup capabilities:

LSP provides:
- go-to-definition
- find-references
- document-symbols
- workspace-symbols

LSP servers exist for: Python, JavaScript, TypeScript, Go, Rust, C/C++, etc.

Option 3: Universal ctags

A lightweight option for symbol indexing:

# Generate tags file
ctags -R --languages=C --fields=+ne --extras=+q

# Lookup symbol
grep "initServer" tags

Practical recommendations

Based on my experience, here’s how to optimize Claude Code for large codebases:

1. Index once, query many times

Setup:
1. Install code-indexing MCP server
2. Run initial index (one-time cost)
3. Configure Claude Code to use MCP tools

Daily workflow:
- Claude Code queries MCP server for symbols
- Tokens saved: 90%+ per query
- Context window preserved for actual work

INEFFICIENT:
Query 1: "Find initServer"     [Cost: Lookup]
Query 2: "Find handleRequest"  [Cost: Lookup]
Query 3: "Find parseConfig"    [Cost: Lookup]
Total: 3 lookups

EFFICIENT:
Query: "Find initServer, handleRequest, and parseConfig"
Total: 1 lookup, 3 definitions returned

3. Keep index updated

#!/bin/bash
# Run as git hook or cron job
find src -name "*.c" -newer .index_timestamp -exec code-index update {} \;
touch .index_timestamp

4. Monitor token savings

Before MCP:
- Query: 10 per session
- Average tokens per query: 50,000
- Total: 500,000 tokens/session

After MCP:
- Query: 10 per session
- Average tokens per query: 5,000
- Total: 50,000 tokens/session

Savings: 450,000 tokens (90%)

Summary

In this post, I explained how to reduce Claude Code token usage by 97% when working with large codebases. The key insight is that Claude Code reads entire files by default, which becomes prohibitively expensive for projects with large files.

The solution is implementing an MCP server that indexes code at the symbol level. Instead of reading a 22,000-line file to understand one function, the MCP server retrieves only the specific definition needed—reducing token consumption from 84,193 to 2,699 in my testing.

To implement this:

Build or adopt a code-indexing MCP server
Pre-index your codebase at the symbol level
Configure Claude Code to query the index instead of reading files
Keep the index updated as your codebase changes

For most developers, using existing tools like Serena or LSP integration is more practical than building from scratch. The token savings make Claude Code viable for enterprise-scale development.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

👨‍💻 Reddit Discussion: Claude Code Token Usage Problem
👨‍💻 Model Context Protocol (MCP) Documentation
👨‍💻 Tree-sitter Parser Generator
👨‍💻 Serena - Code Indexing Tool

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!