Skip to content

What is OpenViking? A Context Database Built for AI Agents

Purpose

This post explains what OpenViking is and how it solves the context management problem for AI Agents.

Problem

When I built my first AI agent, I ran into a frustrating problem: context management. My agent needed to remember conversations, access documentation, and apply learned skills. But there was no unified way to handle all of this.

Here’s what I tried first:

context-fragmentation.txt
Memory -> Stored in code, scattered across variables
Resources -> Vector database, flat embeddings
Skills -> JSON files, hard to discover
Session data -> Lost after each conversation

This fragmentation led to five critical problems:

  1. Context Fragmentation: Memories in code, resources in vector DB, skills everywhere
  2. Context Explosion: Long-running agents generate massive context; truncation loses information
  3. Poor Retrieval Quality: Flat RAG storage lacks global perspective
  4. Context Opacity: Traditional RAG retrieval is a black box
  5. Limited Memory Iteration: Systems only record user memories, not agent task memories

I needed a better solution. That’s when I found OpenViking.

What is OpenViking?

OpenViking is an open-source context database designed specifically for AI Agents. It unifies memory, resources, and skills into a single virtual filesystem paradigm.

The key insight: instead of flat storage like traditional RAG, OpenViking uses hierarchical directory structures. This matches how humans organize information - not as a pile of documents, but as folders within folders.

From the OpenViking README:

“OpenViking is an open-source Context Database designed specifically for AI Agents. We aim to define a minimalist context interaction paradigm for Agents, allowing developers to completely say goodbye to the hassle of context management.”

Environment

  • Python 3.10+
  • OpenViking SDK
  • PostgreSQL (for persistence)
  • OpenAI or Anthropic API (for embeddings)

Core Concepts

Filesystem Paradigm

OpenViking organizes all context using viking:// URIs. Here’s the structure:

filesystem-structure.txt
viking://
├── memory/ # Agent memories (auto-extracted)
│ ├── user/ # User-provided memories
│ └── agent/ # Task execution memories
├── resources/ # External resources (PDFs, docs)
│ └── my_project/
│ ├── api.pdf
│ └── guide.md
└── skills/ # Agent capabilities
└── email/
├── compose.md
└── summarize.md

This structure lets agents navigate context like a filesystem:

filesystem-operations.py
from openviking import OpenViking
client = OpenViking()
# List resources
client.ls("viking://resources/")
# View tree structure
client.tree("viking://resources/my_project", depth=2)
# Read specific file
content = client.read("viking://skills/email/compose.md")

Tiered Context Loading

OpenViking uses three context tiers to prevent context explosion:

context-tiers.txt
L0 (Abstract) -> High-level summaries, fits in system prompt
L1 (Overview) -> Section summaries, retrieved on demand
L2 (Detail) -> Full content, retrieved only when needed

This tiered approach solves the context explosion problem. Instead of loading everything at once, the agent loads L0 first, then drills down as needed.

tiered-loading.py
# L0: Get abstract (always loaded)
abstract = client.get_abstract("viking://resources/api.pdf")
# L1: Get overview (loaded on first query)
overview = client.get_overview("viking://resources/api.pdf")
# L2: Get detail (loaded when agent needs specifics)
detail = client.get_detail("viking://resources/api.pdf", section="authentication")

Directory Recursive Retrieval

Traditional RAG uses vector search only. OpenViking combines vector search with hierarchical browsing:

recursive-retrieval.py
# Intent analysis first
intent = client.analyze_intent("How do I authenticate with the API?")
# Hierarchical search based on intent
results = client.find(
query="authentication methods",
path="viking://resources/api_project",
recursive=True
)
# Results include the retrieval path
for result in results:
print(f"Path: {result.path}")
print(f"Tier: {result.tier}")
print(f"Relevance: {result.score}")

This gives the agent both relevant content AND context about where it came from.

Visualized Retrieval Trajectory

One of my favorite features: OpenViking shows exactly how it retrieved information:

retrieval-trace.txt
Query: "authentication methods"
├── Step 1: Intent analysis -> "Looking for auth docs"
├── Step 2: Navigate to viking://resources/api_project/
├── Step 3: Vector search found 3 matches in auth/
├── Step 4: Loaded L1 overviews for auth/*.md
└── Step 5: Retrieved L2 for auth/oauth.md (highest relevance)
Total latency: 45ms
Tokens used: 1,234

This transparency helps me debug why the agent retrieved certain content.

Automatic Session Management

OpenViking automatically extracts memories from agent sessions:

session-management.py
# Start a session
session = client.create_session("customer_support")
# Agent works...
response = agent.process("I need to return order #12345")
# OpenViking automatically extracts:
# - User has order #12345
# - User wants to return it
# - Agent suggested refund process
# Later, retrieve relevant memories
memories = client.find_relevant_memories("return order")
# Returns memory about order #12345 return request

Practical Example: Building a Documentation Agent

Let me show you how I built a documentation agent with OpenViking.

Step 1: Initialize OpenViking

init-openviking.py
from openviking import OpenViking
from openviking.agents import DocumentAgent
# Initialize client
client = OpenViking(
persist_dir="./viking_data",
embedding_model="text-embedding-3-small"
)
# Create agent
agent = DocumentAgent(client)

Step 2: Add Resources

add-resources.py
# Add documentation
client.add_resource(
url="https://docs.example.com/api.pdf",
path="viking://resources/api_docs"
)
# Add a skill
client.add_skill(
name="search_docs",
description="Search documentation for answers",
template="""
Search the documentation for: {query}
Return the most relevant sections.
""",
path="viking://skills/search"
)

Step 3: Use the Agent

use-agent.py
# Ask a question
response = agent.query("How do I authenticate with OAuth?")
# Response includes retrieval trace
print(response.answer)
print(f"Sources: {response.sources}")
print(f"Trace: {response.retrieval_trace}")

I got this output:

agent-output.txt
Answer: To authenticate with OAuth, you need to:
1. Register your application to get client_id and client_secret
2. Redirect users to the authorization URL
3. Handle the callback to exchange code for token
Sources:
- viking://resources/api_docs/auth/oauth.md (L2)
- viking://resources/api_docs/auth/overview.md (L1)
Trace:
- Intent: Authentication question -> OAuth focus
- Navigate: viking://resources/api_docs/auth/
- Found: oauth.md, overview.md
- Loaded: L2 for oauth.md (highest relevance)

Comparison: OpenViking vs Traditional RAG

I tested both approaches for the same documentation agent:

FeatureTraditional RAGOpenViking
StorageFlat embeddingsHierarchical filesystem
RetrievalVector search onlyVector + path-based navigation
Context tiersAll-or-nothingL0/L1/L2 tiered loading
Retrieval visibilityBlack boxFull trajectory trace
Memory managementManualAutomatic extraction
Session persistenceLost after sessionPreserved and queryable

The key difference: OpenViking treats context like a filesystem, not a flat pile of embeddings.

When to Use OpenViking

OpenViking is ideal when:

  1. Your agent needs long-term memory: Sessions span days or weeks
  2. You have hierarchical resources: Documentation, codebases, knowledge bases
  3. You need retrieval transparency: Debug why agents retrieve certain content
  4. Context explosion is a problem: Agents generate massive context over time
  5. You want automatic memory: No manual memory management

When traditional RAG might be better:

  1. Simple document search: Just need to find relevant documents
  2. Flat data only: No natural hierarchy in your data
  3. No session state: Stateless queries only

Getting Started

Install OpenViking:

Terminal
pip install openviking
# Initialize a project
viking init my-agent
cd my-agent

Create your first agent:

my-agent/main.py
from openviking import OpenViking
client = OpenViking()
# Add a resource
client.add_resource("https://docs.example.com/guide.pdf")
# Search
results = client.find("installation steps")
for result in results:
print(f"Found in: {result.path}")
print(f"Content: {result.content[:200]}...")

Summary

In this post, I explained what OpenViking is and how it solves the context management problem for AI Agents. The key point is that OpenViking uses a filesystem paradigm instead of flat RAG storage, with tiered context loading (L0/L1/L2) and automatic memory extraction.

OpenViking addresses five critical problems: context fragmentation, context explosion, poor retrieval quality, context opacity, and limited memory iteration. If you’re building AI agents that need long-term memory and structured knowledge access, OpenViking is worth exploring.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!

Comments