What Is Headroom and How Does It Reduce LLM Token Costs?

Jun 3, 2026

Abstract data compression

Problem

I was running a coding agent that searches our codebase before answering questions. A single search returned 100 results, and the agent sent all of them to the LLM. That one call cost 17,765 input tokens. At $3 per million tokens, that is about $0.05 per search. With 50 searches a day, the bill adds up fast.

Worse, when the agent also includes build logs, file contents, and conversation history, context windows overflow. The model either rejects the request or truncates important context.

What Is Headroom?

Headroom is a context compression layer that sits between your application and the LLM provider. It intercepts messages, detects the content type — JSON arrays, code, logs, text — and routes each to the best compressor before sending anything to the model.

It works in three modes:

Python/TypeScript SDK — call compress() directly in your code
Proxy mode — run headroom proxy --port 8787 and point any OpenAI-compatible client at it
MCP server — expose headroom_compress and headroom_retrieve tools to Claude Code or Cursor

What Happened When I Tried It

I installed Headroom with pip:

pip install "headroom-ai[all]"

Then I compressed the same 100 search results:

from headroom import compress

result = compress(messages, model="gpt-4o")
print(f"Before: {result.tokens_before}")   # 17765
print(f"After:  {result.tokens_after}")    # 1408
print(f"Saved:  {result.tokens_saved}")    # 16357
print(f"Ratio:  {result.compression_ratio:.0%}")  # 92%

The output was immediate. 17,765 tokens became 1,408 tokens — a 92% reduction. The LLM still answered the question correctly because the compressor kept the important items and dropped the noise.

How the Compression Works

Headroom does not just truncate. It uses different strategies for different content:

SmartCrusher for JSON arrays — statistically scores each item and keeps a representative subset while preserving errors and anomalies
CCR (Compress-Cache-Retrieve) — stores the original locally with a hash key, so the LLM can retrieve the full data if needed
CacheAligner — moves dynamic content like dates out of the system prompt prefix so provider KV caches actually hit

This means you get aggressive compression without permanent data loss. If the model needs the original 100 search results, it calls headroom_retrieve(hash=...) and gets them back in about 1ms.

Real Numbers

The Headroom repository publishes benchmark results:

Workload	Before	After	Savings
Code search (100 results)	17,765	1,408	92%
SRE incident debugging	65,694	5,118	92%
GitHub issue triage	54,174	14,761	73%
Codebase exploration	78,502	41,254	47%

Accuracy is preserved too. GSM8K math scores stayed at 0.870, and TruthfulQA actually improved slightly from 0.530 to 0.560.

Why I Care

For a team running agents daily, 90% token reduction is the difference between a $500 monthly bill and a $50 monthly bill. The proxy mode requires zero code changes — just change the base URL and everything works the same, but cheaper.

Summary

In this post, I showed what Headroom is and how it compresses LLM context. The key point is that Headroom achieves 60-95% token reduction on real workloads while preserving answer accuracy, and it works as a library, proxy, or MCP server.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

👨‍💻 Headroom GitHub Repository
👨‍💻 Headroom Documentation

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!