memsearch vs LangChain memory: Which is better for AI agents?
The Problem
Every AI agent needs persistent memory. The question is: how should you store it?
When I started building agents, I struggled with this choice. I tried LangChain’s built-in memory, then discovered memsearch, and got confused about which one to use. Documentation didn’t give me clear guidance—it just listed features without explaining trade-offs.
Here’s what I learned: neither solution is universally better. The right choice depends entirely on what you prioritize.
Two Storage Paradigms
Before comparing solutions, understand there are two fundamental approaches to agent memory:
DATABASE-FIRST (LangChain)├── Vector store IS the source of truth├── Data stored as opaque binary blobs├── Requires database tools to inspect└── Migration = extract + transform + import
MARKDOWN-FIRST (memsearch)├── Markdown files ARE the source of truth├── Vector DB is rebuildable index (like database index)├── Human-readable, edit with any text editor└── Migration = copy .md filesThe key difference: markdown-first treats vector databases as indexes—derived, rebuildable, and optional. Database-first treats them as primary storage.
This architectural choice drives every other difference between the solutions.
LangChain Memory Deep Dive
LangChain’s memory system has evolved significantly. The old approach with ConversationBufferMemory and VectorStoreMemory is now deprecated in favor of LangGraph’s checkpointing and store APIs.
Here’s how LangGraph handles memory today:
In-Memory Checkpointing:
from langgraph.checkpoint.memory import InMemorySaverfrom langgraph.graph import StateGraph
checkpointer = InMemorySaver()
def call_model(state): response = model.invoke(state["messages"]) return {"messages": response}
builder = StateGraph()builder.add_node(call_model)graph = builder.compile(checkpointer=checkpointer)
config = {"configurable": {"thread_id": "1"}}result = graph.stream({"messages": [{"role": "user", "content": "hi! I'm bob"}]}, config)This works for development and testing. The catch: memory disappears on restart.
PostgreSQL Checkpointing:
from langgraph.checkpoint.postgres import PostgresSaver
DB_URI = "postgresql://postgres:postgres@localhost:5442/postgres"with PostgresSaver.from_conn_string(DB_URI) as checkpointer: graph = builder.compile(checkpointer=checkpointer) checkpointer.setup()This gives you durable, ACID-compliant storage. But you need to run PostgreSQL and set it up.
Redis-backed Memory:
from langgraph.checkpoint.redis import RedisSaverfrom langgraph.store.redis import RedisStore
DB_URI = "redis://localhost:6379"with RedisStore.from_conn_string(DB_URI) as store: with RedisSaver.from_conn_string(DB_URI) as checkpointer: graph = builder.compile( checkpointer=checkpointer, store=store )Fast and scalable, but adds infrastructure complexity.
What LangChain Does Well:
- Deep ecosystem integration (LangChain, LangGraph, LangSmith)
- Multiple persistence backends (memory, PostgreSQL, Redis, filesystem)
- Thread-based conversation tracking across stateful interactions
- Human-in-the-loop patterns for debugging
- Battle-tested in production environments
What LangChain Struggles With:
- Vendor lock-in to LangChain ecosystem
- Opaque storage—I couldn’t easily see what my agent “remembered”
- Multiple overlapping memory classes created confusion during learning
- Migration requires data extraction and transformation
- Debugging stored data requires database queries
I found myself stuck when I needed to understand why my agent was retrieving wrong context. With LangChain’s PostgreSQL-backed memory, I had to write SQL queries just to inspect what was stored.
memsearch Deep Dive
memsearch takes a completely different approach. Memory lives in markdown files you can read, edit, and version control:
from memsearch import MemSearch
mem = MemSearch(paths=["./memory"])await mem.index()results = await mem.search("Redis config", top_k=3)print(results[0]["content"], results[0]["score"])The architecture flow:
Markdown files (.md) ↓File watcher detects changes ↓Content chunking (heading/paragraph) ↓SHA-256 deduplication ↓Embedding generation (OpenAI/Google/Voyage/Ollama) ↓Vector store (Milvus Lite/Server/Zilliz Cloud) ↓Semantic search with rerankingYour actual memory file looks like this:
## User Preferences- Favorite color: blue- Prefers concise responses- No emojis in output
## Redis ConfigurationHost: localhost:6379DB: 0Connection timeout: 5000msYou can edit this directly in any text editor. When my agent started giving wrong answers about Redis config, I just opened the markdown file, corrected the settings, and the fix was immediately available.
What memsearch Does Well:
- Zero vendor lock-in—markdown files are portable
- Human-readable and debuggable without tools
- Git-friendly version control built-in
- Simple API with fewer lines of code
- Framework agnostic—works with any LLM or agent framework
- Local-only option with Ollama for privacy
What memsearch Struggles With:
- Different paradigm that can feel “primitive” coming from database-first approaches
- Manual file management for some advanced workflows
- Newer project with smaller community
- File system operations at scale require planning
Full Agent Comparison
Here’s how the same agent looks with each solution:
LangChain with LangGraph:
from langgraph.checkpoint.postgres import PostgresSaverfrom langgraph.prebuilt import create_react_agentfrom langchain_openai import ChatOpenAI
model = ChatOpenAI(model="gpt-4o-mini")checkpointer = PostgresSaver.from_conn_string(DB_URI)
agent = create_react_agent(model, checkpointer=checkpointer)config = {"configurable": {"thread_id": "1"}}result = agent.invoke({"messages": [{"role": "user", "content": "What's our Redis config?"}]}, config)memsearch with custom agent:
import asynciofrom datetime import datefrom pathlib import Pathfrom openai import OpenAIfrom memsearch import MemSearch
MEMORY_DIR = "./memory"llm = OpenAI()mem = MemSearch(paths=[MEMORY_DIR])
def save_memory(content: str): p = Path(MEMORY_DIR) / f"{date.today()}.md" p.parent.mkdir(parents=True, exist_ok=True) with open(p, "a") as f: f.write(f"\n{content}\n")
async def agent_chat(user_input: str) -> str: # 1. Recall relevant memories memories = await mem.search(user_input, top_k=3) context = "\n".join(f"- {m['content'][:200]}" for m in memories)
# 2. Generate response with context resp = llm.chat.completions.create( model="gpt-4o-mini", messages=[ {"role": "system", "content": f"You have these memories:\n{context}"}, {"role": "user", "content": user_input}, ], ) answer = resp.choices[0].message.content
# 3. Store new memory save_memory(f"## {user_input}\n{answer}") await mem.index()
return answerThe memsearch version is more explicit about the recall-think-remember pattern, but gives you full control. The LangChain version is more integrated but hides the details.
Framework Integration Comparison
When you’re working with different agent frameworks, integration complexity varies:
| Framework | LangChain Integration | memsearch Integration |
|---|---|---|
| LangChain | Native memory classes | BaseRetriever pattern |
| LangGraph | Checkpointer + Store | Tool wrapper |
| LlamaIndex | Custom vector store | Custom retriever |
| CrewAI | No native integration | Tool wrapper |
| Standalone | N/A | Direct API (minimal) |
memsearch wins on consistency—the same API pattern works across all frameworks. With LangChain, each framework requires different approaches.
Quick Decision Matrix
Here’s when to use each solution:
| Use Case | Recommended Solution | Reasoning |
|---|---|---|
| Quick prototype | memsearch | 3-line setup, no infrastructure |
| LangChain ecosystem project | LangChain Memory | Native integration, less friction |
| Multi-framework project | memsearch | Consistent API across frameworks |
| Debug-heavy development | memsearch | Readable markdown files |
| Production with LangGraph | LangChain Memory | Proven checkpointer patterns |
| Team with git workflows | memsearch | Native version control |
| Privacy-critical app | memsearch with Ollama | Local-only operation |
| Need data migration | memsearch | Export/import markdown files |
| Stateful conversations | LangChain Memory | Thread-based tracking |
| Multi-agent system | memsearch | Shared memory across agents |
Performance Comparison
LangChain Memory Performance:
- InMemorySaver: Fastest, but not persistent
- PostgresSaver: Durable, moderate latency (~10-50ms)
- RedisSaver: Fastest persistent option (~1-10ms)
- FilesystemMiddleware: Variable, depends on FS performance
memsearch Performance:
- Milvus Lite: In-memory, good for development (< 1GB data)
- Milvus Server: Production-ready, scalable to millions of vectors
- Zilliz Cloud: Managed service, auto-scaling
- Hybrid search: Dense + sparse + RRF reranking for better relevance
For most applications, performance differences won’t be noticeable unless you’re operating at significant scale.
Migration Strategies
The markdown-first approach makes migration trivial:
From LangChain to memsearch:
- Extract memories from your checkpointing backend
- Write them as markdown files organized by topic or date
- Run
await mem.index()to build the vector index
From memsearch to LangChain:
- Copy your markdown files
- Parse them into LangChain’s expected format
- Ingest into the chosen checkpointing backend
With memsearch, you always have your raw data in markdown. With LangChain, you need database-specific export tools.
LlamaIndex and CrewAI Notes
These frameworks have their own memory systems:
LlamaIndex:
from llama_index.core.memory import Memoryfrom llama_index.core.memory import VectorMemory, ChatMemoryBuffer
memory = Memory.from_defaults(session_id="my_session", token_limit=40000)LlamaIndex uses composable memory patterns—vector memory for long-term, chat buffer for short-term. memsearch replaces the vector memory component while giving you transparency.
CrewAI:
from crewai import Agent, Crew, Task
agent = Agent(role="...", goal="...", memory=True)crew = Crew(agents=[agent], tasks=[...], memory=True)CrewAI’s built-in memory is simple but opaque. I use memsearch when I need to inspect or version control what my crews learn.
Common Pitfalls
LangChain Memory Pitfalls:
- Choosing InMemorySaver for production (data loss on restart)
- Not persisting thread IDs correctly (state gets lost)
- Confusion from multiple overlapping memory classes
- Debugging stored data requires database queries
memsearch Pitfalls:
- Chunking too aggressively (loses context between chunks)
- Not re-indexing after manual file edits
- Choosing wrong embedding model (cost vs. quality tradeoff)
- Forgetting file system permissions in production
The Reason
I think the real question isn’t “which is better?” but “what paradigm fits your workflow?”
Database-first systems (LangChain) excel when:
- You’re already committed to an ecosystem
- You value framework integration over transparency
- You have DevOps capacity for database management
Markdown-first systems (memsearch) excel when:
- You want to understand and control what your agent knows
- You prefer simple over integrated
- You value portability and debuggability
The architectural choice—database vs markdown as source of truth—drives everything else.
Summary
memsearch and LangChain Memory solve the same problem with different paradigms. Neither is universally better. LangChain excels for ecosystem integration and production provenance. memsearch excels for transparency, portability, and framework flexibility.
Choose LangChain Memory when you’re deeply invested in the LangChain ecosystem and need battle-tested checkpointing patterns.
Choose memsearch when you value human-readable, version-controlled memory and want zero vendor lock-in.
Final Words + More Resources
My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me
Here are also the most important links from this article along with some further resources that will help you in this scope:
- 👨💻 memsearch - A Markdown-first memory system
- 👨💻 LangChain Memory Concepts
- 👨💻 LangGraph Persistence & Checkpointing
- 👨💻 LangGraph Memory Store
- 👨💻 LlamaIndex Memory Guide
- 👨💻 CrewAI Agent Memory
- 👨💻 Chroma Agentic Memory
Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!
Comments