OpenViking L0/L1/L2 Context Layers: How Tiered Loading Reduces Token Costs
Problem
I was burning through tokens at an alarming rate. My AI agent system was loading entire documentation libraries into every request, even for simple queries. A 50k token context for a “what’s the API endpoint for user creation?” question was wasteful beyond belief.
Query: "How do I authenticate with the API?"Context loaded: 47,000 tokens of documentationTokens actually needed: ~500 tokensWaste: 98.9%Cost per query: $0.70 instead of $0.007I knew I needed progressive context loading, but implementing it from scratch felt daunting. Then I discovered OpenViking’s L0/L1/L2 context layer system.
The Problem with Naive Context Loading
Most RAG systems suffer from the same issue: they retrieve document chunks but don’t consider the granularity of what’s needed. You either get too little context (missing key information) or too much (wasting tokens on irrelevant details).
┌─────────────────────────────────────────────────────────┐│ Traditional RAG Approach ││ ││ Query: "OAuth implementation" ││ ││ ┌─────────────────────────────────────────────────┐ ││ │ Retrieved Documents │ ││ │ │ ││ │ [Chunk 1: OAuth intro - 800 tokens] │ ││ │ [Chunk 2: JWT section - 1200 tokens] │ ││ │ [Chunk 3: API keys - 900 tokens] │ ││ │ [Chunk 4: Rate limiting - 1100 tokens] │ ││ │ [Chunk 5: Error codes - 700 tokens] │ ││ │ ... │ ││ │ │ ││ │ Total: 15,000+ tokens loaded for every query │ ││ └─────────────────────────────────────────────────┘ ││ ││ Problem: No granularity control. All or nothing. │└─────────────────────────────────────────────────────────┘The agent loads everything because it doesn’t know what’s relevant until it reads it. But reading costs tokens.
Enter OpenViking’s Three-Tier Model
OpenViking solves this by automatically processing content into three distinct layers:
┌─────────────────────────────────────────────────────────┐│ OpenViking Context Layers ││ ││ ┌─────────────────────────────────────────────────┐ ││ │ L0: Abstract (~100 tokens) │ ││ │ • Ultra-short summary │ ││ │ • For vector search & quick filtering │ ││ │ • "API authentication guide covering OAuth..." │ ││ └─────────────────────────────────────────────────┘ ││ ↓ ││ ┌─────────────────────────────────────────────────┐ ││ │ L1: Overview (~2k tokens) │ ││ │ • Comprehensive summary with navigation │ ││ │ • Section summaries guide to L2 content │ ││ │ • Usually sufficient for decision-making │ ││ └─────────────────────────────────────────────────┘ ││ ↓ ││ ┌─────────────────────────────────────────────────┐ ││ │ L2: Detail (unlimited) │ ││ │ • Full original content │ ││ │ • Loaded only when absolutely necessary │ ││ │ • On-demand retrieval │ ││ └─────────────────────────────────────────────────┘ ││ ││ Load L0 first, then L1 if needed, then L2. │└─────────────────────────────────────────────────────────┘This hierarchical approach means you only pay for what you need.
L0: The Abstract Layer
The L0 abstract is an ultra-condensed summary, typically around 100 tokens. I use this for initial relevance filtering and vector search.
API authentication guide covering OAuth 2.0, JWT tokens, and API keys for secure access. Includes implementation examples for each method.When I search for “OAuth”, the system can quickly scan L0 abstracts to identify relevant documents without loading their full content.
from openviking import OpenViking
client = OpenViking()
# L0 abstracts enable fast vector searchresults = client.find("OAuth implementation")
for ctx in results.resources: # Each result has an L0 abstract pre-loaded print(f"Abstract: {ctx.abstract}") # Output: "API authentication guide covering OAuth 2.0..."The L0 layer answers: “Is this document relevant to my query?”
L1: The Overview Layer
The L1 overview is where the real savings happen. At around 1-2k tokens, it provides comprehensive summaries with navigation guidance.
# Authentication Guide Overview
This guide covers three authentication methods for the API:
## Sections- **OAuth 2.0** (L2: oauth.md): Complete OAuth flow with code examples- **JWT Tokens** (L2: jwt.md): Token generation and validation- **API Keys** (L2: api-keys.md): Simple key-based authentication
## Key Points- OAuth 2.0 recommended for user-facing applications- JWT for service-to-service communication- API keys suitable for simple integrations
## Quick StartFor most use cases, start with OAuth 2.0 using the client credentials flow.I discovered that L1 is usually sufficient for most agent decisions. The overview tells me enough to understand what’s in the full document and whether I need to dive deeper.
# Get overview for top resultsfor ctx in results.resources[:3]: overview = client.overview(ctx.uri) print(f"Overview: {overview[:200]}...")
# Check if overview contains what we need if "client_credentials" in overview: print("Found client credentials flow in overview!") # We might not even need L2The L1 layer answers: “What’s in this document and should I read more?”
L2: The Detail Layer
L2 is the full original content, loaded only when absolutely necessary. This is where you go for deep-dive operations.
# Only load L2 if overview doesn't have enough detailif needs_full_content: full_content = client.read(ctx.uri) # Now we have the complete documentationThe key insight: most queries never reach L2. The L1 overview contains enough context for 80-90% of decisions.
┌─────────────────────────────────────────────────────────┐│ Typical Query Resolution ││ ││ Query: "How to implement OAuth client credentials?" ││ ││ Step 1: Vector search over L0 abstracts ││ → Found auth guide (relevance: 0.92) ││ → Cost: ~100 tokens ││ ││ Step 2: Load L1 overview ││ → Found "client_credentials" mentioned ││ → Overview explains the flow ││ → Cost: ~2,000 tokens ││ ││ Step 3: Decision - is L2 needed? ││ → Overview has enough detail ││ → No need to load full content ││ → Cost: 0 additional tokens ││ ││ Total: 2,100 tokens vs 47,000 tokens ││ Savings: 95.5% │└─────────────────────────────────────────────────────────┘How OpenViking Generates L0/L1
The magic happens during resource ingestion. OpenViking automatically generates L0 and L1 layers:
Input Document → Parser → TreeBuilder → AGFS → SemanticQueue → L0/L1 Generation │ ▼ ┌─────────────────────────┐ │ Generation Process │ │ │ │ 1. Parse document │ │ 2. Build content tree │ │ 3. Generate L0 abstract │ │ 4. Generate L1 overview │ │ 5. Store L2 original │ │ │ │ Bottom-up: leaves first │ │ Hierarchical aggregation│ └─────────────────────────┘The generation follows a bottom-up approach: leaf nodes are processed first, then parent directories. This allows child L0s to inform parent L1s.
Directory Structure After Processing:
viking://resources/docs/auth/├── .abstract.md # L0: ~100 tokens├── .overview.md # L1: ~1k tokens├── .relations.json # Related resources├── oauth.md # L2: Full content├── jwt.md # L2: Full content└── api-keys.md # L2: Full contentThe hidden .abstract.md and .overview.md files store the generated layers.
Real-World Token Savings
I tested this with my documentation system. The results were dramatic:
┌─────────────────────────────────────────────────────────┐│ Token Consumption Comparison │├─────────────────────────────────────────────────────────┤│ ││ Traditional Approach: ││ ┌───────────────────────────────────────────────────┐ ││ │ Query → Load all chunks → Process → Response │ ││ │ │ ││ │ Average context per query: 45,000 tokens │ ││ │ Average cost per query: $0.68 │ ││ │ 100 queries/day cost: $68 │ ││ └───────────────────────────────────────────────────┘ ││ ││ OpenViking L0/L1/L2: ││ ┌───────────────────────────────────────────────────┐ ││ │ Query → L0 search → L1 overview → (maybe L2) │ ││ │ │ ││ │ Average context per query: 3,200 tokens │ ││ │ Average cost per query: $0.048 │ ││ │ 100 queries/day cost: $4.80 │ ││ │ │ ││ │ L2 loaded in only 12% of queries │ ││ └───────────────────────────────────────────────────┘ ││ ││ Token reduction: 92.9% ││ Cost reduction: 92.9% ││ Monthly savings: $1,896 │└─────────────────────────────────────────────────────────┘The OpenClaw integration results confirm this: 91-96% reduction in input token cost with improved task completion.
When to Use Each Layer
Through experimentation, I developed this decision matrix:
┌─────────────────────────────────────────────────────────┐│ Layer Selection Guide │├─────────────────────────┬───────────────────────────────┤│ Scenario │ Recommended Layer │├─────────────────────────┼───────────────────────────────┤│ Quick relevance check │ L0 (abstract) ││ Broad topic search │ L0 → L1 ││ Understand scope │ L1 (overview) ││ Decision-making │ L1 (usually sufficient) ││ Code extraction │ L2 (detail) ││ Detailed implementation │ L2 (detail) ││ Building LLM context │ L1 (start here, add L2 if ││ │ specific sections needed) │└─────────────────────────┴───────────────────────────────┘Progressive Loading in Practice
Here’s how I implemented progressive loading in my agent system:
from openviking import OpenVikingfrom typing import List, Dict, Optional
class ProgressiveContextLoader: """ Load context progressively through L0 → L1 → L2 layers. Only loads what's needed for each query. """
def __init__(self): self.client = OpenViking()
def find_relevant_resources(self, query: str, max_results: int = 5) -> List: """Step 1: Use L0 for initial filtering.""" results = self.client.find(query) return results.resources[:max_results]
def get_overviews(self, resources: List) -> List[Dict]: """Step 2: Load L1 overviews for decision-making.""" overviews = [] for resource in resources: overview = self.client.overview(resource.uri) overviews.append({ "uri": resource.uri, "abstract": resource.abstract, # L0 "overview": overview, # L1 "needs_l2": False }) return overviews
def assess_l2_needs(self, overviews: List[Dict], query: str) -> List[Dict]: """Step 3: Determine which resources need L2.""" keywords = self._extract_keywords(query)
for item in overviews: # Check if overview has enough detail overview_text = item["overview"].lower()
# Simple heuristic: if query keywords are well-covered coverage = sum(1 for kw in keywords if kw in overview_text)
if coverage < len(keywords) * 0.5: item["needs_l2"] = True
return overviews
def load_l2_content(self, items: List[Dict]) -> List[Dict]: """Step 4: Load L2 only for resources that need it.""" for item in items: if item["needs_l2"]: item["full_content"] = self.client.read(item["uri"]) else: item["full_content"] = None
return items
def build_context(self, query: str) -> str: """Full progressive loading pipeline.""" # L0: Find relevant resources resources = self.find_relevant_resources(query)
# L1: Get overviews items = self.get_overviews(resources)
# Assess L2 needs items = self.assess_l2_needs(items, query)
# L2: Load full content only where needed items = self.load_l2_content(items)
# Build final context context_parts = [] for item in items: if item["full_content"]: context_parts.append(item["full_content"]) else: context_parts.append(item["overview"])
return "\n\n---\n\n".join(context_parts)
def _extract_keywords(self, query: str) -> List[str]: """Extract key terms from query.""" # Simple keyword extraction stopwords = {"how", "what", "where", "when", "why", "the", "a", "an", "is", "are"} words = query.lower().split() return [w for w in words if w not in stopwords and len(w) > 2]
# Usageloader = ProgressiveContextLoader()context = loader.build_context("How to implement OAuth client credentials flow?")This pipeline ensures we only load L2 content when the L1 overview doesn’t provide sufficient detail.
Best Practices
Based on my experience, here’s what I recommend:
1. Start with L0 for search
# WRONG: Load everythingdocs = load_all_documents() # Expensive!
# RIGHT: Use L0 for initial filteringresults = client.find(query) # Fast, cheap2. L1 is usually enough
# For most queries, L1 overview is sufficientoverview = client.overview(uri)
# Only proceed to L2 if truly neededif needs_deep_dive(overview): full = client.read(uri)3. Monitor L2 load rates
# Track how often you load L2l2_load_rate = l2_loads / total_queries
# If > 20%, your L1 generation might need improvement# If < 5%, you might be missing important context4. Use L1 for context building
# WRONG: Stuff L2 into promptscontext = client.read(uri) # Could be 50k+ tokens
# RIGHT: Start with L1, add L2 sections as neededcontext = client.overview(uri) # ~2k tokensif specific_section_needed: context += client.read(uri + "/section.md")Summary
In this post, I showed how OpenViking’s L0/L1/L2 tiered context model reduces token consumption by 80-90% compared to naive context loading.
The key insight is progressive loading: start small (L0 abstracts for search), load more when needed (L1 overviews for decisions), and only pull full content (L2 details) for deep-dive operations. This transforms context economics from “pay for everything upfront” to “pay as you go.”
To implement tiered loading in your AI applications:
- Use L0 abstracts for vector search and quick filtering
- Load L1 overviews for decision-making (usually sufficient)
- Reserve L2 full content for on-demand deep dives
- Monitor your L2 load rate as a health metric
Your token budget will thank you.
Final Words + More Resources
My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me
Here are also the most important links from this article along with some further resources that will help you in this scope:
Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!
Comments