Pi Agent + DeepSeek: Why the Harness Can Matter More Than the Model

Sep 1, 2026

pi agent logo

Problem

Two coding agents can use the same underlying model and still produce very different success rates, token usage, and costs.

I saw this clearly when I started testing the Pi Agent (earendil-works/pi) with DeepSeek V4 Flash. The same DeepSeek model that felt sluggish and expensive inside one harness felt faster, cheaper, and more reliable inside another.

Most agent marketing focuses on the model. If you believe that, switching models looks like the only lever on quality and cost. But the more I watched these tools behave differently on identical tasks, the clearer one idea became: the harness is part of the inference system. Changing the harness changes what the model sees and how it acts, so the same model can look smarter or dumber depending on what wraps it.

I call this effect the Harness Multiplier. This post explains what it is, why Pi + DeepSeek is a good example of it, and how to set up the pairing yourself.

The benchmark that started the discussion

Diagram showing how the same DeepSeek model can produce different success rates, token usage, and costs across AI coding agent harnesses

The recent developer chatter started with a community benchmark: the same DeepSeek model, run across several agent harnesses on the same task set, produced visible gaps in success rate and cost.

The table below is illustrative. It comes from developer-run benchmark discussion, so the exact numbers need to be checked against the original Composio benchmark dataset before you quote them as fact. What matters here is the shape of the result:

Harness	Successful Tasks	Approx. Cost per Successful Task
Pi	20 / 30	~$0.028
Oh My Pi	17 / 30	—
Claude Code	~16 / 30	~$0.195
Codex	~16 / 30	—
Other harness	…	…

Provenance note: model version, task count, success definition, harness list, and test date must all be confirmed against the original benchmark before this table is treated as evidence. Whatever the final numbers are, the interesting result is not that Pi “won” one benchmark. It is that changing the harness changed the behavior of the same underlying model.

What is an AI coding agent harness?

AI coding agent architecture showing the model surrounded by system prompts, tools, context management, retries, and session state

Here is the single most important concept in this post:

A harness is not merely a UI around the model. It actively changes what the model sees and how the model acts.

A coding agent is not a model. It is a system:

Model
+ System Prompt
+ Tool Definitions
+ Tool Execution Loop
+ Context Management
+ Compaction
+ Retry Strategy
+ File Editing Strategy
+ Session State
= Coding Agent

DeepSeek V4 is just the model. Pi, Claude Code, Codex, OpenCode, and DeepSeek Harness decide:

what prompt the model sees
which tools it can call
how much tool output is returned
how files are edited and how shell commands run
how failures are retried
when context gets compacted
how sessions are saved

All of that happens before the model even sees a single token.

Why the same model feels smarter in a different harness

Harness Multiplier diagram explaining how prompts, tools, context, retries, and compaction affect AI coding agent performance

This is the causal chain that explains the Harness Multiplier:

Harness
  ↓
system prompt
  ↓
tool selection
  ↓
observations returned
  ↓
context length
  ↓
next-token distribution
  ↓
agent behavior

There is a second chain, about recovery rather than generation:

Harness
  ↓
retry / editing / compaction strategy
  ↓
probability of recovering from mistakes

I think the easiest analogy is hardware:

Model = CPU

Harness = Operating System + Runtime

The same CPU produces very different application-level results depending on the OS and runtime around it. If the runtime schedules badly, leaks memory, or feeds the CPU bad instructions, the CPU looks slow. The model is the CPU; the harness is everything else.

That is why the era is shifting. In 2024–2025 the question was “which model is best?” In 2026 it is increasingly “which model + harness combination is best for this workload?”

Why Pi is so minimal

Pi is built around a deliberately small core. The main built-in tools stay close to the basics:

read
write
edit
bash

Compared with a large agent harness, this means:

fewer tool schemas the model has to understand
a smaller system prompt
fewer harness-specific rules to follow

The practical effects are less instruction interference, less token overhead, and a more stable prompt prefix. I found the last point matters most for caching (more on that below).

One warning before I go further: do not conclude that “small prompts always perform better.” The accurate statement is the opposite of a universal rule:

Minimal prompts can be especially useful for models that were not specifically optimized for a large proprietary harness.

Open models like DeepSeek were not trained against Claude Code’s prompt format. Feeding them a clean, minimal harness can reduce confusion that a big proprietary prompt would cause.

DeepSeek prompt caching and Pi

DeepSeek prompt caching diagram showing how Pi Agent's stable system prompt and tool definitions can create a reusable cached prefix

DeepSeek’s context caching is the economic engine behind this pairing. According to DeepSeek’s official docs:

the caching service runs automatically for all users, with no code changes required
a cache hit only happens when a request’s prefix exactly matches a previously persisted prefix starting from the 0th token
partial matches in the middle of an input do not trigger a cache hit
responses expose prompt_cache_hit_tokens and prompt_cache_miss_tokens in the usage block

So caching rewards stable prefixes. The first request looks like this:

System prompt
+ tools
+ project instructions
+ conversation history
+ new request

The next request looks like this:

System prompt
+ tools
+ project instructions
+ conversation history
+ new tool result

The big prefix is almost identical. If the harness keeps that prefix stable — stable system prompt, stable tool schemas, append-only context — the cache hit rate can get very high.

Here is the simplified cost model from the brief. Without cache:

50,000 tokens × full input price

With a 95% cached prefix:

47,500 cached tokens  (cache-read price)
+ 2,500 uncached tokens (full input price)

That is why the biggest Pi + DeepSeek advantage may not be the raw model price. It may be how effectively the harness allows the provider’s cache to work.

One honest caveat: some developers report cache hit rates above 99% in long-running, highly repetitive sessions. Treat that as workload-specific anecdotal evidence, not a guarantee for every repository and session.

Pi’s session tree and context engineering

Pi Agent session tree compared with a linear coding agent conversation, showing branching, rollback, and summarized failed paths

Many coding agents are effectively one long linear conversation:

A → B → C → D → E

Pi’s session tree is closer to a real branching structure:

A
├── B
│   ├── C
│   └── D
└── E
    └── F

You can branch, roll back to an earlier node, bookmark a spot, try an alternate path, and compact context. A concrete example: I asked the agent to fix an authentication bug.

Path A: it changed token validation → tests failed → I rolled back to the previous node
Path B: it checked the middleware → found the real issue → fixed it

The failed Path A was summarized as:

Changing token validation did not fix the bug and broke X tests.

So the new path keeps the valuable information without permanently stuffing the whole failed transcript into context. For long-running tasks, experimentation, and debugging, this behaves more like managing working memory than like extending a chat transcript forever.

Self-extending agents: Pi extensions

Pi’s most unusual feature is that it does not push every capability into the core. When you need something new, you can:

write a TypeScript extension
load the new tool or behavior
let the agent help you create the extension
use it in the current workflow

Traditional approach: the core product keeps adding tools. Pi approach: keep the core small and move specialization outward.

A practical scenario: I told the agent:

Create a tool that queries our internal REST API and exposes it as check_build_status.

The agent generated the extension, registered the tool, and then called it. Pi treats extensibility as something the agent can participate in, rather than something only the harness developer controls.

How to set up Pi with DeepSeek

The following steps match the current Pi docs at the time of writing. Verify against the pinned Pi version you install, since config formats evolve.

Step 1 — Install Pi

npm install -g --ignore-scripts @earendil-works/pi-coding-agent

Step 2 — Store your DeepSeek API key

Provider credentials live in ~/.pi/agent/auth.json with 0600 permissions:

{
  "deepseek": { "type": "api_key", "key": "sk-..." }
}

You can also use OpenRouter, Cloudflare AI Gateway, or any other OpenAI-compatible endpoint as the provider instead of the direct DeepSeek API.

Step 3 — Configure the model

Here is a minimal DeepSeek provider block for models.json:

{
  "providers": {
    "deepseek": {
      "baseUrl": "https://api.deepseek.com",
      "api": "openai-completions",
      "apiKey": "$DEEPSEEK_API_KEY",
      "models": [
        {
          "id": "deepseek-v4-flash",
          "name": "DeepSeek V4 Flash",
          "contextWindow": 1000000,
          "maxTokens": 384000,
          "input": ["text"],
          "reasoning": true,
          "cost": {
            "input": 0.14,
            "output": 0.28,
            "cacheRead": 0.028,
            "cacheWrite": 0
          }
        }
      ]
    }
  }
}

Add deepseek-v4-pro analogously with its own id, name, and pricing. The exact cost numbers and context windows should be re-checked against the live docs before you rely on them.

Step 4 — Start Pi

pi --provider deepseek --model deepseek-v4-flash

Step 5 — Verify what is active

Use the auth check command to confirm the provider and model are ready:

pi auth check --provider deepseek --model deepseek-v4-flash

For cache visibility, look at the usage block in API responses: prompt_cache_hit_tokens tells you how many tokens hit the cache, and prompt_cache_miss_tokens tells you how many did not.

How to improve DeepSeek cache hit rates

Here are seven practical rules I follow:

Keep the system prompt stable.
Avoid constantly rewriting global instructions.
Put stable repository instructions early.
Avoid unnecessary tool-schema churn.
Reuse long-running sessions where appropriate.
Use compaction instead of repeatedly rebuilding context.
Avoid injecting timestamps or random dynamic content into the prompt prefix unless you actually need it.

And the warning that matters most: do not sacrifice correctness, context freshness, security, or task isolation just to chase a higher cache hit rate. A cheap wrong answer is still wrong.

Pi vs DeepSeek Harness vs Claude Code vs OpenCode

Different products optimize for different goals. Here is the feature-level view (verify current versions before making decisions):

Feature	Pi	DeepSeek Harness	Claude Code	OpenCode
Design philosophy	Minimal core	Plugin/runtime oriented	Integrated coding product	Open extensible agent
Core tool count	Very small	varies	larger	configurable
System prompt	Minimal	depends	larger/private	depends
Multi-provider	Strong	DeepSeek-oriented	primarily Anthropic ecosystem	strong
DeepSeek support	Yes	Native focus	limited/non-primary	Yes
Session branching	Strong	verify	verify	verify
Extensions	TypeScript	plugin ecosystem	hooks/skills	plugins
SDK / embedding	Strong	verify	limited	strong
Prompt cache friendliness	High potential	high	provider dependent	config dependent
Best for	developers building their own harness workflow	DeepSeek power users	polished integrated coding	multi-provider users

Who wins depends on what you need:

Pi fits developers who want to control the agent runtime, use open models, work across multiple providers, embed an agent via SDK, or write their own extensions.
Claude Code fits developers who want a mature coding product out of the box and are happy inside the Anthropic ecosystem.
OpenCode fits multi-provider terminal users who want a more finished experience than a minimal harness.
DeepSeek Harness is DeepSeek-first, plugin/runtime oriented, with an expanding ecosystem.

One question worth sitting with: is DeepSeek Harness actually competing with Pi, or are both converging on the same idea — keep the model powerful while making the runtime programmable?

Benchmark caveats

One benchmark cannot prove that “Pi is universally better than Claude Code.” Success rates shift with model version, provider, temperature, retry policy, tool schema, timeout, task type, benchmark scoring, context size, and harness version. A 30-task sample is small.

The defensible conclusion is narrower:

Harness choice can materially change model performance, and Pi appears particularly competitive with DeepSeek in this benchmark.

Community and X posts are anecdotes. Treat them as such, and never write them up as objective facts.

Who should use Pi + DeepSeek?

Good fit:

developers experimenting with open models
developers optimizing API cost
DeepSeek API users
multi-provider users
developers building custom coding agents
teams wanting auditable agent behavior
developers embedding agents into other applications

Probably not the best fit:

developers who want a zero-configuration product
teams that depend heavily on polished proprietary integrations
non-technical users
developers unwilling to manage extensions and configuration themselves

Final takeaway

Models get most of the attention, but agent performance is increasingly becoming a systems problem. The harness controls the prompt, the tools, the observations, the retries, the compaction, and the session state — and all of those change the model’s behavior.

The shareable version:

Don’t benchmark the model alone. Benchmark the model, harness, tools, context strategy, and cache behavior as one system.

Summary

In this post, I explained why the same DeepSeek model performs differently across coding-agent harnesses, and why that is a systems problem rather than a model problem. The key point is that the harness is part of the inference system: with Pi’s minimal core, session tree, and stable context, DeepSeek’s automatic prompt caching has a better chance to work — which is how the same model can end up cheaper and more reliable.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

👨‍💻 Pi Coding Agent (earendil-works/pi)
👨‍💻 Pi Docs: Quickstart
👨‍💻 Pi Docs: Model Configuration (models.json)
👨‍💻 Pi Docs: Providers
👨‍💻 Pi Docs: Compaction and Session Tree
👨‍💻 DeepSeek API: Context Caching
👨‍💻 DeepSeek API: Caching Service
👨‍💻 DeepSeek API: Pricing
👨‍💻 DeepSeek API: V4 Flash Announcement

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!