Pi-Herdr
A durable multi-agent orchestration extension for Pi and Herdr, with a SQLite control plane, durable mailboxes, workflow DAGs and structured completion.
- Role
- Author
- Status
- Active
- Stack
- TypeScript
- Repository
- github.com/pwnholic/pi-herdr
Overview
Pi-Herdr is an extension for Pi and Herdr that adds multi-agent orchestration: a SQLite control plane, communication between workers, durable mailboxes, workflow DAGs, recovery, and a structured completion mechanism.
Of everything I have built on my own time, this is the project closest to the problems I work on professionally - coordinating independent processes that can fail independently of each other.
Problem
Running several agents on one task is easy to start and hard to finish. Without a place that records what was asked, what has been attempted and what came back, the work exists only in message history: a crash loses it, a retry duplicates it, and there is no way to tell whether a step succeeded or merely stopped producing output.
Constraints
- Durability over convenience. State that matters has to survive a restart, which rules out keeping coordination in memory.
- No new infrastructure. A single-file SQLite database keeps the project installable as an extension rather than a deployment.
- Bounded failure handling. Recovery has to know what was in flight, so a worker cannot be treated as “either running or gone”.
Architecture
The control plane is a SQLite database. Workers communicate through durable mailboxes rather than direct calls, and workflows are expressed as DAGs. Recovery reads the state that was persisted, which is what makes a restarted worker resumable rather than re-runnable.
Technical decisions
| Decision | Alternative | Why |
|---|---|---|
| SQLite control plane | In-memory coordination | A restart must not lose the workflow’s state |
| Durable mailboxes | Direct worker-to-worker calls | A message that was accepted has to survive the sender exiting |
| Workflow DAGs | Sequential step list | Independent steps should be able to run at the same time |
| Structured completion | “Process exited” | Exit status alone cannot say whether a step produced a usable result |
Tradeoffs
A single-file control plane trades horizontal scalability for operational simplicity: it is the right shape for a local orchestration tool and the wrong one for a multi-host system. The build-time cost of writing down every state transition is also real - it is slower to extend than a design that keeps coordination in memory, and it is the reason recovery works.
Results
The repository documents this functionality and passes its own checks. I am not quoting throughput or reliability numbers here, because I have not measured them in a way that would justify a figure.
Lessons learned
Writing the state down before doing the work is what makes recovery possible; a recovery path designed after the fact is usually a retry path with extra steps.