Open source

Pi-Herdr

A durable multi-agent orchestration extension for Pi and Herdr, with a SQLite control plane, durable mailboxes, workflow DAGs, and structured completion.

Role
Author
Status
Active
Stack
TypeScript

What this project shows

Context

Personal open-source project exploring durable coordination for multi-agent systems.

My contribution

Author and maintainer, from the control-plane model through durable messaging and workflow recovery.

Why it matters

A concrete example of designing around durable state, independent failure, and resumable work.

Public evidence

Highlights

  • SQLite control plane for agent identity, assignment, and state
  • Durable mailboxes between parent and worker agents
  • Persistent workflow DAGs with recovery after a restart
  • Structured completion instead of process exit status

Durable Multi-Agent Orchestration - SQLite Control Plane - Durable Mailboxes - Workflow DAGs

Overview

Pi-Herdr is an extension for Pi and Herdr that adds multi-agent orchestration: a SQLite control plane, communication between workers, durable mailboxes, workflow DAGs, recovery, and a structured completion mechanism.

Of everything I have built on my own time, this is the project closest to the problems I work on professionally - coordinating independent processes that can fail independently of each other.

Problem

Running several agents on one task is easy to start and hard to finish. Without a place that records what was asked, what has been attempted and what came back, the work exists only in message history: a crash loses it, a retry duplicates it, and there is no way to tell whether a step succeeded or merely stopped producing output.

Constraints

  • Durability over convenience. State that matters has to survive a restart, which rules out keeping coordination in memory.
  • No new infrastructure. A single-file SQLite database keeps the project installable as an extension rather than a deployment.
  • Bounded failure handling. Recovery has to know what was in flight, so a worker cannot be treated as “either running or gone”.

Architecture

The control plane is a SQLite database. Workers communicate through durable mailboxes rather than direct calls, and workflows are expressed as DAGs. Recovery reads the state that was persisted, which is what makes a restarted worker resumable rather than re-runnable.

A parent and its workers exchange durable mail through mailboxes, a SQLite control plane records identities, workflows, and structured completions, while recovery reads that state to resume work after a restart.

Mermaid preview

Technical decisions

DecisionAlternativeWhy
SQLite control planeIn-memory coordinationA restart must not lose the workflow’s state
Durable mailboxesDirect worker-to-worker callsA message that was accepted has to survive the sender exiting
Workflow DAGsSequential step listIndependent steps should be able to run at the same time
Structured completion“Process exited”Exit status alone cannot say whether a step produced a usable result

Tradeoffs

A single-file control plane trades horizontal scalability for operational simplicity: it is the right shape for a local orchestration tool and the wrong one for a multi-host system. The build-time cost of writing down every state transition is also real - it is slower to extend than a design that keeps coordination in memory, and it is the reason recovery works.

Results

The repository documents this functionality and passes its own checks. I am not quoting throughput or reliability numbers here, because I have not measured them in a way that would justify a figure.

Lessons learned

Writing the state down before doing the work is what makes recovery possible; a recovery path designed after the fact is usually a retry path with extra steps.

Verify

The repository is public and self-checking:

bun install --frozen-lockfile
bun run check   # tsc --noEmit, node --test, biome format check, oxlint

Search this site