What is OpenViking? A Context Database Built for AI Agents
Purpose
This post explains what OpenViking is and how it solves the context management problem for AI Agents.
Problem
When I built my first AI agent, I ran into a frustrating problem: context management. My agent needed to remember conversations, access documentation, and apply learned skills. But there was no unified way to handle all of this.
Here’s what I tried first:
Memory -> Stored in code, scattered across variablesResources -> Vector database, flat embeddingsSkills -> JSON files, hard to discoverSession data -> Lost after each conversationThis fragmentation led to five critical problems:
- Context Fragmentation: Memories in code, resources in vector DB, skills everywhere
- Context Explosion: Long-running agents generate massive context; truncation loses information
- Poor Retrieval Quality: Flat RAG storage lacks global perspective
- Context Opacity: Traditional RAG retrieval is a black box
- Limited Memory Iteration: Systems only record user memories, not agent task memories
I needed a better solution. That’s when I found OpenViking.
What is OpenViking?
OpenViking is an open-source context database designed specifically for AI Agents. It unifies memory, resources, and skills into a single virtual filesystem paradigm.
The key insight: instead of flat storage like traditional RAG, OpenViking uses hierarchical directory structures. This matches how humans organize information - not as a pile of documents, but as folders within folders.
From the OpenViking README:
“OpenViking is an open-source Context Database designed specifically for AI Agents. We aim to define a minimalist context interaction paradigm for Agents, allowing developers to completely say goodbye to the hassle of context management.”
Environment
- Python 3.10+
- OpenViking SDK
- PostgreSQL (for persistence)
- OpenAI or Anthropic API (for embeddings)
Core Concepts
Filesystem Paradigm
OpenViking organizes all context using viking:// URIs. Here’s the structure:
viking://├── memory/ # Agent memories (auto-extracted)│ ├── user/ # User-provided memories│ └── agent/ # Task execution memories├── resources/ # External resources (PDFs, docs)│ └── my_project/│ ├── api.pdf│ └── guide.md└── skills/ # Agent capabilities └── email/ ├── compose.md └── summarize.mdThis structure lets agents navigate context like a filesystem:
from openviking import OpenViking
client = OpenViking()
# List resourcesclient.ls("viking://resources/")
# View tree structureclient.tree("viking://resources/my_project", depth=2)
# Read specific filecontent = client.read("viking://skills/email/compose.md")Tiered Context Loading
OpenViking uses three context tiers to prevent context explosion:
L0 (Abstract) -> High-level summaries, fits in system promptL1 (Overview) -> Section summaries, retrieved on demandL2 (Detail) -> Full content, retrieved only when neededThis tiered approach solves the context explosion problem. Instead of loading everything at once, the agent loads L0 first, then drills down as needed.
# L0: Get abstract (always loaded)abstract = client.get_abstract("viking://resources/api.pdf")
# L1: Get overview (loaded on first query)overview = client.get_overview("viking://resources/api.pdf")
# L2: Get detail (loaded when agent needs specifics)detail = client.get_detail("viking://resources/api.pdf", section="authentication")Directory Recursive Retrieval
Traditional RAG uses vector search only. OpenViking combines vector search with hierarchical browsing:
# Intent analysis firstintent = client.analyze_intent("How do I authenticate with the API?")
# Hierarchical search based on intentresults = client.find( query="authentication methods", path="viking://resources/api_project", recursive=True)
# Results include the retrieval pathfor result in results: print(f"Path: {result.path}") print(f"Tier: {result.tier}") print(f"Relevance: {result.score}")This gives the agent both relevant content AND context about where it came from.
Visualized Retrieval Trajectory
One of my favorite features: OpenViking shows exactly how it retrieved information:
Query: "authentication methods"├── Step 1: Intent analysis -> "Looking for auth docs"├── Step 2: Navigate to viking://resources/api_project/├── Step 3: Vector search found 3 matches in auth/├── Step 4: Loaded L1 overviews for auth/*.md└── Step 5: Retrieved L2 for auth/oauth.md (highest relevance)
Total latency: 45msTokens used: 1,234This transparency helps me debug why the agent retrieved certain content.
Automatic Session Management
OpenViking automatically extracts memories from agent sessions:
# Start a sessionsession = client.create_session("customer_support")
# Agent works...response = agent.process("I need to return order #12345")
# OpenViking automatically extracts:# - User has order #12345# - User wants to return it# - Agent suggested refund process
# Later, retrieve relevant memoriesmemories = client.find_relevant_memories("return order")# Returns memory about order #12345 return requestPractical Example: Building a Documentation Agent
Let me show you how I built a documentation agent with OpenViking.
Step 1: Initialize OpenViking
from openviking import OpenVikingfrom openviking.agents import DocumentAgent
# Initialize clientclient = OpenViking( persist_dir="./viking_data", embedding_model="text-embedding-3-small")
# Create agentagent = DocumentAgent(client)Step 2: Add Resources
# Add documentationclient.add_resource( url="https://docs.example.com/api.pdf", path="viking://resources/api_docs")
# Add a skillclient.add_skill( name="search_docs", description="Search documentation for answers", template=""" Search the documentation for: {query} Return the most relevant sections. """, path="viking://skills/search")Step 3: Use the Agent
# Ask a questionresponse = agent.query("How do I authenticate with OAuth?")
# Response includes retrieval traceprint(response.answer)print(f"Sources: {response.sources}")print(f"Trace: {response.retrieval_trace}")I got this output:
Answer: To authenticate with OAuth, you need to:1. Register your application to get client_id and client_secret2. Redirect users to the authorization URL3. Handle the callback to exchange code for token
Sources:- viking://resources/api_docs/auth/oauth.md (L2)- viking://resources/api_docs/auth/overview.md (L1)
Trace:- Intent: Authentication question -> OAuth focus- Navigate: viking://resources/api_docs/auth/- Found: oauth.md, overview.md- Loaded: L2 for oauth.md (highest relevance)Comparison: OpenViking vs Traditional RAG
I tested both approaches for the same documentation agent:
| Feature | Traditional RAG | OpenViking |
|---|---|---|
| Storage | Flat embeddings | Hierarchical filesystem |
| Retrieval | Vector search only | Vector + path-based navigation |
| Context tiers | All-or-nothing | L0/L1/L2 tiered loading |
| Retrieval visibility | Black box | Full trajectory trace |
| Memory management | Manual | Automatic extraction |
| Session persistence | Lost after session | Preserved and queryable |
The key difference: OpenViking treats context like a filesystem, not a flat pile of embeddings.
When to Use OpenViking
OpenViking is ideal when:
- Your agent needs long-term memory: Sessions span days or weeks
- You have hierarchical resources: Documentation, codebases, knowledge bases
- You need retrieval transparency: Debug why agents retrieve certain content
- Context explosion is a problem: Agents generate massive context over time
- You want automatic memory: No manual memory management
When traditional RAG might be better:
- Simple document search: Just need to find relevant documents
- Flat data only: No natural hierarchy in your data
- No session state: Stateless queries only
Getting Started
Install OpenViking:
pip install openviking
# Initialize a projectviking init my-agentcd my-agentCreate your first agent:
from openviking import OpenViking
client = OpenViking()
# Add a resourceclient.add_resource("https://docs.example.com/guide.pdf")
# Searchresults = client.find("installation steps")
for result in results: print(f"Found in: {result.path}") print(f"Content: {result.content[:200]}...")Summary
In this post, I explained what OpenViking is and how it solves the context management problem for AI Agents. The key point is that OpenViking uses a filesystem paradigm instead of flat RAG storage, with tiered context loading (L0/L1/L2) and automatic memory extraction.
OpenViking addresses five critical problems: context fragmentation, context explosion, poor retrieval quality, context opacity, and limited memory iteration. If you’re building AI agents that need long-term memory and structured knowledge access, OpenViking is worth exploring.
Final Words + More Resources
My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me
Here are also the most important links from this article along with some further resources that will help you in this scope:
Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!
Comments