How to Coordinate Multiple AI Coding Agents Effectively: Supervisor Patterns and Distributed Systems Principles

Mar 12, 2026

“Agent coordination is a distributed systems problem wearing a trenchcoat.”

That’s what I discovered after spending two weeks trying to orchestrate multiple Claude Code agents to work on a substantial codebase refactoring project. What started as an exciting experiment in parallel AI development quickly descended into chaos: duplicate work, conflicting changes, agents silently crashing, and tasks claimed but never completed.

The Problem: Why My Agent Team Failed

I started with a simple premise: give each agent a git worktree, assign them independent tasks, and let them work in parallel. The supervisor agent would coordinate everything. Sounds reasonable, right?

Here’s what actually happened:

Day 1:
  - Agent A and Agent B both claimed "implement authentication middleware"
  - Agent C crashed silently while running tests
  - Agent D's lease on the database migration task never expired
  - Agent E reported "task complete" but tests were failing

Day 2:
  - I discovered 6 instances of the same utility function
  - Three agents had conflicting implementations
  - Two tasks were "completed" with TODO comments everywhere
  - One agent was stuck waiting for a dependency that was already done

The root cause? I was treating agents as reliable teammates rather than fallible distributed processes.

Distributed Systems Principles Apply

The same failure modes that plague distributed systems appeared in my agent team:

Distributed Systems Problem	Agent Coordination Equivalent
Stale locks	Agent claims task forever after crash
Zombie processes	Agent crashes but lease remains
Race conditions	Multiple agents claim same task
Byzantine failures	Agent reports success but work is incomplete
Network partitions	Agent cannot communicate with supervisor

The insight that changed everything: I needed to build proper distributed systems primitives.

The Solution: Supervisor + Lease + Heartbeat + Verification

After studying how distributed databases handle similar problems, I implemented a coordination layer with four critical components.

1. Lease-Based Task Claiming

Tasks are claimed with a time-limited lease, not an infinite lock:

from dataclasses import dataclass
from datetime import datetime, timedelta
from typing import Optional

@dataclass
class Task:
    id: str
    description: str
    dependencies: list[str]  # Task IDs that must complete first
    status: str  # "pending", "claimed", "in_progress", "pending_verification", "complete"
    claimed_by: Optional[str] = None
    lease_expiry: Optional[datetime] = None
    last_heartbeat: Optional[datetime] = None

class Coordinator:
    def __init__(self, lease_duration: timedelta = timedelta(minutes=30)):
        self.tasks: dict[str, Task] = {}
        self.agents: dict[str, Agent] = {}
        self.lease_duration = lease_duration

    def claim_task(self, agent_id: str, task_id: str) -> bool:
        """Atomically claim a task with a lease."""
        task = self.tasks.get(task_id)
        if not task or task.status != "pending":
            return False

        # Check dependencies are complete
        for dep_id in task.dependencies:
            if self.tasks[dep_id].status != "complete":
                return False

        # Acquire lease
        task.status = "claimed"
        task.claimed_by = agent_id
        task.lease_expiry = datetime.now() + self.lease_duration
        task.last_heartbeat = datetime.now()

        return True

This prevents the “stale lock” problem: if an agent crashes, the lease expires and another agent can pick up the work.

2. Heartbeat Mechanism

Agents must send periodic heartbeats to prove they’re still alive:

from datetime import datetime, timedelta

class Coordinator:
    def __init__(self):
        self.heartbeat_timeout = timedelta(minutes=10)

    def heartbeat(self, agent_id: str) -> bool:
        """Renew lease and confirm agent is alive."""
        agent = self.agents.get(agent_id)
        if not agent:
            return False

        agent.last_heartbeat = datetime.now()

        # Renew task lease if working
        if agent.current_task:
            task = self.tasks[agent.current_task]
            task.last_heartbeat = datetime.now()
            task.lease_expiry = datetime.now() + self.lease_duration

        return True

    def check_agent_health(self):
        """Mark agents as unhealthy if no heartbeat."""
        now = datetime.now()
        for agent in self.agents.values():
            if now - agent.last_heartbeat > self.heartbeat_timeout:
                agent.status = "unhealthy"
                # Release their task
                if agent.current_task:
                    task = self.tasks[agent.current_task]
                    task.status = "pending"
                    task.claimed_by = None

This catches the “zombie agent” problem: an agent that crashed without releasing its task.

3. Verification Gates

The most surprising discovery was that agents frequently reported completion when work was incomplete. Verification gates solved this:

class VerificationGate:
    """Independent verification before task is considered complete."""

    def __init__(self):
        self.checks = [
            self.no_todos_remaining,
            self.no_unused_imports,
            self.tests_pass,
            self.type_check_passes,
            self.code_is_wired,
        ]

    def verify(self, task: Task, modified_files: list[str]) -> tuple[bool, list[str]]:
        """Run all verification checks. Returns (passed, failures)."""
        failures = []
        for check in self.checks:
            result, message = check(task, modified_files)
            if not result:
                failures.append(f"{check.__name__}: {message}")

        return len(failures) == 0, failures

    def no_todos_remaining(self, task, files) -> tuple[bool, str]:
        """Check no TODO comments in modified files."""
        for file_path in files:
            content = read_file(file_path)
            if "TODO" in content or "FIXME" in content:
                return False, f"Found TODO/FIXME in {file_path}"
        return True, ""

    def tests_pass(self, task, files) -> tuple[bool, str]:
        """Run related tests."""
        result = subprocess.run(["pytest", task.test_file], capture_output=True)
        if result.returncode != 0:
            return False, f"Tests failed: {result.stdout.decode()}"
        return True, ""

    def code_is_wired(self, task, files) -> tuple[bool, str]:
        """Verify code is actually imported/used, not orphaned."""
        # Check that new functions are called somewhere
        # Check that routes are registered
        # Check that exports are imported
        pass

Tasks enter pending_verification state after an agent claims completion. Only after independent verification passes is the task marked complete.

4. MCP-Based Messaging

Agents need to communicate with the supervisor without failing their tasks:

class AgentMessaging:
    """MCP-based communication channel for agents."""

    async def send_progress(self, agent_id: str, task_id: str, progress: dict):
        """Report progress to supervisor."""
        await self.mcp_client.call("supervisor/progress", {
            "agent_id": agent_id,
            "task_id": task_id,
            "progress": progress,
            "timestamp": datetime.now().isoformat()
        })

    async def request_clarification(self, agent_id: str, task_id: str, question: str):
        """Ask supervisor for clarification."""
        response = await self.mcp_client.call("supervisor/clarify", {
            "agent_id": agent_id,
            "task_id": task_id,
            "question": question
        })
        return response.get("answer")

    async def report_blocker(self, agent_id: str, task_id: str, blocker: str):
        """Report a blocker without failing the task."""
        await self.mcp_client.call("supervisor/blocker", {
            "agent_id": agent_id,
            "task_id": task_id,
            "blocker": blocker
        })

This enables escalation without task failure, allowing the supervisor to provide guidance or reassign work.

The Supervisor Pattern

The supervisor agent has a specific responsibility: decompose work into independent tasks that can be parallelized without blocking each other.

┌─────────────────────────────────────────────────────────────┐
│                      SUPERVISOR AGENT                       │
│                                                             │
│  1. Analyze epic/codebase                                   │
│  2. Identify independent components                         │
│  3. Create task graph with dependencies                     │
│  4. Assign tasks to workers                                 │
│  5. Monitor heartbeats and lease expiry                    │
│  6. Reassign failed/stuck tasks                            │
│  7. Run verification gates                                  │
│  8. Merge results and handle conflicts                     │
└─────────────────────────────────────────────────────────────┘
                           │
          ┌────────────────┼────────────────┐
          │                │                │
          ▼                ▼                ▼
    ┌──────────┐     ┌──────────┐     ┌──────────┐
    │ Worker A │     │ Worker B │     │ Worker C │
    │          │     │          │     │          │
    │ ─Heartbeat─►   │ ─Heartbeat─►   │ ─Heartbeat─►
    │ ─Progress──►   │ ─Progress──►   │ ─Progress──►
    │ ─Done──────►   │ ─Done──────►   │ ─Done──────►
    └──────────┘     └──────────┘     └──────────┘

Poor task decomposition creates the worst failure mode: tightly-coupled tasks that block each other, defeating parallelism.

What I Got Wrong Initially

Looking back, my first attempts had several critical mistakes:

No verification: I trusted agents when they reported “done”. Big mistake. Agents frequently marked tasks complete with failing tests, TODO comments, or code that was never wired into the application.

Infinite leases: Tasks were claimed forever. When an agent crashed, that task was locked indefinitely.

No heartbeats: I couldn’t detect stuck or crashed agents. They just disappeared, leaving orphaned locks.

Poor decomposition: The supervisor created tasks like “implement feature X” without considering that multiple features might touch the same files. Agents spent more time resolving conflicts than writing code.

No communication channel: Agents couldn’t ask for clarification. They either made assumptions (often wrong) or failed the task entirely.

Why This Matters

Single-agent workflows hit a ceiling: context limits, sequential execution, no parallelism. Multi-agent coordination unlocks parallel work, but only with proper distributed systems primitives.

The supervisor quality directly determines project success. A good supervisor:

Decomposes work into truly independent tasks
Orders tasks to maximize parallelism
Handles failures gracefully
Provides clear guidance when agents need help

Without these mechanisms, agent teams create more chaos than output.

Common Anti-Patterns to Avoid

Anti-Pattern	Consequence	Solution
No supervisor	Agents duplicate work, create conflicts	Central coordinator with clear authority
Trusting status reports	False completions pollute codebase	Independent verification gates
Infinite leases	Dead tasks block work forever	Time-limited leases with expiry
No heartbeats	Cannot detect stuck agents	Periodic heartbeat requirement
Poor decomposition	Tasks block each other	Analyze dependencies before assignment
Missing messaging	Agents fail on ambiguity	Escalation channel without task failure
Ignoring protocol violations	Agents skip steps	Verification gates catch shortcuts

The Reality Check

After implementing all these mechanisms, my agent team became significantly more reliable. But the Reddit discussion I referenced hit on an important point: supervisor quality still varies with epic complexity.

Simple refactoring? The coordination works well. Complex architectural changes? The supervisor struggles to decompose work cleanly, and agents spend time waiting for clarification or resolving conflicts.

Agent coordination is still an emerging discipline. We’re building on decades of distributed systems knowledge, but AI agents introduce unique challenges: they hallucinate, they misunderstand instructions, they take shortcuts.

The coordination layer I’ve described doesn’t solve those problems. It creates a safety net that catches failures and prevents cascading damage.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!