diff --git a/durable-agent-runtime/analysis.md b/durable-agent-runtime/analysis.md index 8ed5c88..1b2e7cf 100644 --- a/durable-agent-runtime/analysis.md +++ b/durable-agent-runtime/analysis.md @@ -3,13 +3,14 @@ > Part of [[durable-agent-runtime/index|Durable Agent Runtime]] > Harness/loop analysis moved to [[harness-engineering/index|Harness Engineering]] (2026-08-11). -## The three primitive families +## The primitive families | Family | Example | Compute primitive | Where state lives | Cold start | Hibernation | | --- | --- | --- | --- | --- | --- | | Durable objects | celld (Deno) | V8 isolate running Worker bundles | Per-object SQLite, replicated to S3 | Fast (isolate) | Idle cells hibernate | | Actors | rivet | Lightweight process (Node/Bun/Deno) | In-memory + SQLite/BYO DB | ~20ms | Actors hibernate when idle | | VM sandboxes | hypeman, E2B | Firecracker/Cloud Hypervisor/QEMU microVM | VM disk (snapshot for standby) | ~30s VM / sub-30ms from snapshot | standby = snapshot to disk | +| Durable harness library | pi-durable (2026) | Node process or Cloudflare Durable Object | Immutable transcript + documents + task checkpoints in SQLite/JSONL/DO storage | Reopen + reconcile (ms) | Task sleeps end the invocation; host alarm wakes (onWake on main, not in 1.3.0) | | Collaborative substrate | Radicle COBs | Signed, CRDT-like Git commit DAGs | Git objects (replicated, self-certifying) | Instant (local git) | Repos persist; nodes sync on demand | (Harness libraries — nanocodex, Codex, pi — moved to [[harness-engineering/index|Harness Engineering]].) @@ -26,6 +27,12 @@ The industry convergence: **compute is disposable, state is durable elsewhere.** 1. **Cold start latency** — the gap between "VM ~30s" and "actor ~20ms" is closed with snapshots (E2B, hypeman standby, celld/rivet hibernation) or pre-provisioned pools. This is why orbs can feel local despite being remote. 2. **State checkpointing & recovery** — thread JSONL (Amp), SQLite replication (celld), actor state persistence (rivet), VM disk snapshots (hypeman). Everything is replayable; nothing is lost on compute failure. +## Where pi-durable fits (2026-10-20) + +`@earendil-works/pi-durable` is not a new substrate; it is a **durable harness library** that runs on an existing one. It commits the loop's own state — immutable transcript entries, typed documents, per-phase task checkpoints, idempotent submissions — and resumes on reopen. The substrate can be a Node process with SQLite/JSONL or a Cloudflare Durable Object; tools execute through a pluggable `ExecutionEnv`, so a VM sandbox (hypeman/E2B) can sit underneath without the harness knowing. + +Its semantics sharpen the "compute disposable, state durable" thesis: the harness process is disposable too, but **phases are at-least-once**, not exactly-once. External effects repeat on rerun unless memoized or idempotent; tools declare `replay: "safe"` to rerun, otherwise the model gets an `interrupted` result. Only submission admission is exactly-once (`requestId`). That is a useful, honest contract to compare against Temporal-style durable execution and against our own kanban/session design. Full write-up: [[durable-agent-runtime/pi-durable|pi-durable]]. + ## What this means for hermes-vm (our stack) Already have (the "durable compute host" layer): @@ -48,3 +55,5 @@ Interesting but optional: ## Verdict Build the **environment contracts** (setup hooks, AGENTS.md, dev-server skills, event wake) on the existing durable host. Do not build the sandbox platform or a distributed state layer. herdr + systemd + kanban + tailscale already provide the durability; the missing piece is making the environment "assume an agent and tell it where the light switches are." + +If we ever build a long-running, multi-client agent *service* (as opposed to cron-driven one-shot tasks), pi-durable is the closest reference implementation: one SQLite-backed Session per service, tools on `NodeExecutionEnv`, views for clients. For the Rust-core direction, treat its spec as the design input and reimplement only the commit/task/ownership semantics we actually need. diff --git a/durable-agent-runtime/index.md b/durable-agent-runtime/index.md index bc27aad..65e55ba 100644 --- a/durable-agent-runtime/index.md +++ b/durable-agent-runtime/index.md @@ -1,12 +1,12 @@ --- title: Durable Agent Runtime -description: State & durability lens of agent infrastructure — durable objects (celld), actors (rivet), multi-hypervisor VM runtimes (hypeman, E2B/Amp Orbs), scale-to-zero/hibernation — and what they mean for building a self-hosted orb-like runtime on hermes-vm. Sibling of harness-engineering under the agent-infra umbrella. +description: State & durability lens of agent infrastructure — durable objects (celld), actors (rivet), multi-hypervisor VM runtimes (hypeman, E2B/Amp Orbs), durable harness libraries (pi-durable), scale-to-zero/hibernation — and what they mean for building a self-hosted orb-like runtime on hermes-vm. Sibling of harness-engineering under the agent-infra umbrella. status: active category: technical -tags: [durable-objects, actors, agent-runtime, agent-infra, sandbox, microvm, self-hosting, orbs, stateful, durable-execution] +tags: [durable-objects, actors, agent-runtime, agent-infra, sandbox, microvm, self-hosting, orbs, stateful, durable-execution, pi-durable] draft: true created: 2026-08-07 -updated: 2026-08-11 +updated: 2026-10-20 origin: buzz://d8a718be-031f-4a6e-9f5c-a55466641654/4123d35532d0b2cbfb80a0a34f79fb3e4e63c16c25b77d43bc0f10bb178e4f1b --- @@ -14,7 +14,7 @@ origin: buzz://d8a718be-031f-4a6e-9f5c-a55466641654/4123d35532d0b2cbfb80a0a34f79 ## Summary -Research into how modern agent execution environments achieve **durability and statefulness** — where agent state lives, what compute primitive runs the agent, and how the environment hibernates and wakes. Covers durable objects / durable execution (celld, rivet, Durable Objects), actor runtimes, and microVM sandboxes (hypeman, E2B/Amp Orbs). The harness/loop lens (nanocodex, pi, Code Mode) now lives in [[harness-engineering/index|Harness Engineering]]. Goal: derive a build plan for a self-hosted, orb-like runtime on our NixOS VM (hermes-vm) without overbuilding. +Research into how modern agent execution environments achieve **durability and statefulness** — where agent state lives, what compute primitive runs the agent, and how the environment hibernates and wakes. Covers durable objects / durable execution (celld, rivet, Durable Objects), actor runtimes, durable harness libraries (pi-durable), and microVM sandboxes (hypeman, E2B/Amp Orbs). The harness/loop lens (nanocodex, pi, Code Mode) now lives in [[harness-engineering/index|Harness Engineering]]. Goal: derive a build plan for a self-hosted, orb-like runtime on our NixOS VM (hermes-vm) without overbuilding. ## Research Question @@ -36,6 +36,7 @@ What are the architectural primitives for a durable, stateful agent runtime (san - **Scale-to-zero / hibernation is universal**: Amp orbs pause after 5 min idle; celld cells hibernate; rivet actors hibernate when idle; hypeman standby = snapshot VM to disk, resume in milliseconds; E2B snapshot reuse gives sub-30ms cold starts. - **Two hard problems everyone solves**: (a) cold start latency (actor ~20ms vs VM ~30s, mitigated by snapshot/pooling), (b) state checkpointing and recovery (SQLite replication, thread JSONL, VM snapshots, actor state persistence). - **For our stack**: hermes-vm is already a durable compute host (systemd services, kanban DB, session DBs, herdr panes). The real gap is a per-task disposable environment contract + event wake, not a distributed state layer. celld/rivet solve problems we don't have on a single box; hypeman is interesting for per-task VMs but likely overkill. +- **A new primitive family emerged (2026-10-01): the durable harness library.** `@earendil-works/pi-durable` commits the agent loop itself — transcript entries, documents, tasks, submissions — and resumes unfinished work on reopen. It is substrate-agnostic: Node + SQLite/JSONL, or one Session per Cloudflare Durable Object. Its semantics are the interesting part: phases are at-least-once, tool effects are opt-in replay (`safe` vs `interrupted`), and only submission admission is exactly-once (`requestId`). See [[durable-agent-runtime/pi-durable|pi-durable]]. ## Sources @@ -45,6 +46,7 @@ For the full reference list see [[durable-agent-runtime/sources|Sources]]. - [[durable-agent-runtime/sources|Sources]] — full reference list - [[durable-agent-runtime/analysis|Analysis]] — taxonomy, comparison matrix, implications for our build +- [[durable-agent-runtime/pi-durable|pi-durable]] — deep dive + crash/replay experiments on `@earendil-works/pi-durable` 1.3.0 (2026-10-20) - [[durable-agent-runtime/agent-collaboration|Agent-Native Collaboration]] — agents as first-class signed participants (Radicle COBs / delegates / namespaces) - [[durable-agent-runtime/next-steps|Next Steps]] — follow-up questions and build plan @@ -64,6 +66,7 @@ Triggered from the Amp Orbs thread (2026-08-07). User connected the dots: orbs, - [ ] Decide Phase 1 scope: per-repo `.agents/setup`-style contracts + AGENTS.md per repo on hermes-vm - [ ] Evaluate hypeman as an opt-in per-task VM runtime (snapshot/standby) vs NixOS containers +- [ ] Decide whether to prototype pi-durable (Node/SQLite) for one long-running multi-client agent service, or only borrow its semantics for Hermes core (see [[durable-agent-runtime/pi-durable|pi-durable]] §Implications) ## Conclusion diff --git a/durable-agent-runtime/next-steps.md b/durable-agent-runtime/next-steps.md index 27cb967..2a5e013 100644 --- a/durable-agent-runtime/next-steps.md +++ b/durable-agent-runtime/next-steps.md @@ -7,6 +7,8 @@ - [ ] Which repos get the `.agents/setup`-style contract first (clan-private, law retrieval, idx-cli)? - [ ] Should the dev-server + ports-file pattern be a shared script library in clan-private or per-repo? - [ ] Is hypeman worth an evaluation build on hermes-vm (per-task VM with standby) vs NixOS containers? +- [ ] pi-durable: prototype one long-running service on Node/SQLite, or borrow only its semantics? (see [[durable-agent-runtime/pi-durable|pi-durable]]) +- [ ] pi-durable open questions: no Postgres/KV adapter; multi-client transport is app-owned; default Node env has no sandbox; no published recovery/latency benchmarks; `sleeping`/`onWake`/`restartLimit` not yet in a published release - [ ] Does the Forgejo webhook → kanban wake need a new service, or does an existing cron/webhook path cover it? - [ ] Code Mode pattern for our Effect TS stack now tracked in [[harness-engineering/index|Harness Engineering]] diff --git a/durable-agent-runtime/pi-durable.md b/durable-agent-runtime/pi-durable.md new file mode 100644 index 0000000..54fbdd1 --- /dev/null +++ b/durable-agent-runtime/pi-durable.md @@ -0,0 +1,213 @@ +# pi-durable — Durable Agent Harness + +> Part of [[durable-agent-runtime/index|Durable Agent Runtime]] +> Researched 2026-10-20 against `earendil-works/pi` main (`48c8f32`, 2026-10-20) and the published npm package `@earendil-works/pi-durable@1.3.0` (2026-10-15). Status: **experimental**, API changes between releases. + +## TL;DR + +`@earendil-works/pi-durable` is an MIT-licensed TypeScript **library that turns an agent harness into a durable state machine**. Conversations, transcript entries, tool calls, model turns, and application state are committed to storage before they are published; a new process opening the same storage resumes unfinished work from per-task checkpoints. + +It is not a sandbox, VM, or durable-object platform. It is the layer *above* the substrate: storage-agnostic (memory / SQLite / JSONL / Cloudflare Durable Object SQLite) and execution-environment-agnostic (local Node, remote VM, container). That makes it the clearest example so far of a distinct layer in this topic's taxonomy: the durable harness library, orthogonal to where compute runs. + +The important caveat: durability here is **at-least-once execution of phases, not exactly-once external effects**. Effects must be memoized or made idempotent by the application. Tool calls get an explicit replay policy (`safe` / `unsafe`); an interrupted unsafe tool is reported to the model as `interrupted`, never silently repeated. + +## Where it sits + +| Layer | Examples | pi-durable's relationship | +| --- | --- | --- | +| Durable object / actor platform | celld, rivet, Cloudflare Durable Objects | pi-durable can *run on* a DO (SQLite-backed) or Node; it does not provide the platform | +| Sandbox / VM runtime | hypeman, E2B, Amp orbs | pi-durable does not isolate tools; an `ExecutionEnv` can point at one | +| Durable harness library | **pi-durable**, Temporal-style durable execution applied to agents | This is the new layer: commits the loop's state, not the compute | +| Harness / loop library | pi, nanocodex, Codex, Code Mode | pi-durable is the durable variant of the harness itself | + +The convergence thesis in [[durable-agent-runtime/analysis|Analysis]] — compute is disposable, state is durable elsewhere — still holds. pi-durable sharpens it: *the harness process is also disposable; the conversation + task graph is the durable object.* + +## Architecture + +### Record model + +The normative spec (`packages/durable/docs/spec.md`, ~5,600 lines) defines six primitives: + +- **Session** — owns one mutation line: conversations, entries, tasks, submissions, documents. Every commit is atomic across all of them. +- **Conversation** — a transcript scope; can fork another at an entry. Root conversation is created on first use and found again after restart. +- **Entry** — one immutable transcript record: `pi.user`, `pi.assistant`, `pi.tool-result`, `pi.system`, `pi.reset`, `pi.compaction`, plus application kinds. Entries are never mutated or reused. +- **Document** — mutable typed JSON state stored next to the transcript, implemented on **Chord** (`@earendil-works/chord`) as an operation log with periodic complete bases. Built-ins: `pi.agent`, `pi.provider`, `pi.live`, `pi.inbox`, `pi.usage`. +- **Task** — a durable state machine with an explicit phase checkpoint. Every step commits before moving on. +- **Submission** — an awaitable input or write; `requestId` makes retries idempotent. + +Required invariants (spec §1) worth quoting because they are the design: + +1. One Session commit is atomic across all record and document writes. +2. A document update is published only after its storage commit succeeds. +3. All visible progress is durable; there is no volatile publication path. +4. External effects do not run inside the Session mutation transaction. +5. Entries and IDs are immutable. +6. Commit/close observers capture immutable state synchronously; user callbacks run later, off the mutation line. +7. A storage error is fatal to the open Session — it publishes nothing and must be reopened. + +### Task model + +A task is `defineTask({ name, version, initial, phases, abort, migrate? })`. Each phase receives `(task, runtime, context)` and ends by committing a replacement checkpoint, a `waiting` state, a sleep, or a terminal outcome. Recovery never resumes a phase mid-way: it reruns the phase from its top, so post-wait work belongs in the next phase and its inputs in the checkpoint. On `main`, a task sleeps by ending its phase (`runtime.sleepUntil(...)` or returning `{ status: "sleeping", until, checkpoint }`); the published 1.3.0 still has the older `runtime.sleep()`, which waits *inside* a phase — removed in `[Unreleased]` precisely because a restart reran the phase and restarted the wait (see version gap below). + +Task statuses in the spec: `pending`, `running`, `waiting` (parked on other tasks, `allSettled` or `failFast`), `completing` (outcome decided, owned work still draining), `terminal`. Main adds `sleeping`; published 1.3.0 does not. + +Built-in tasks: + +- `pi.generation` — prepares the system prompt, calls the model, owns the round's `pi.tool` tasks, hands the run to the next generation. +- `pi.tool` — one call per task; intent is committed before `execute()`. +- `pi.compaction` — summarizes older entries; runs as a task, can run in background while the conversation continues. + +Usage/cost is not a task: it is the built-in `pi.usage` document, incremented by generation, tools, and compaction, with subtree accounting for owned conversations. + +### Structured concurrency + +Tasks and conversations form one ownership tree (spec §5.5): + +- Every task names its owner: its conversation or another task. Owner edges are immutable. +- **Abort flows down.** A live owner's cancellation intent (abort mark or held non-`completed` outcome) marks all ordinary owned work; abort handlers run bottom-up so each task undoes its own effects. +- **Owned work does not outlive its owner's finish.** A task that finishes while owned work is live becomes `completing`; terminal only after the subtree drains. +- **Background work is a boundary**: it survives the parent's abort and does not keep the parent busy (used for persistent subagents). +- This is the same shape as structured concurrency in languages (Kotlin/Nursery), applied to agent tasks and subagent conversations. + +### Durability semantics — the part that matters + +The core pattern is the **effect sandwich** (spec §5.2): + +```text +commit intent phase +perform external effect +commit outcome or next phase +``` + +Reopening in an intent phase means the effect *may* have happened. The handler must retry safely, poll an external handle, or record interruption. There is no magic. + +Concrete guarantees, per mechanism: + +- **Submissions**: a `requestId` deduplicates admission within a conversation. A client can retry after a crash and get the original submission back. This is the cleanest exactly-once primitive in the package. +- **Model turns**: streaming partials are committed at most every `settings.progress.partialIntervalMs` (default 100 ms). On recovery, a leftover partial is converted to an **aborted** `pi.assistant` entry and the request is resent from the same committed context, with the same model, thinking level, stream options, and provider `sessionId` (prompt cache/session affinity). So: no token-level resume; duplicate model spend is possible; the partial answer survives in the transcript as aborted. +- **Tool calls**: intent (final arguments + replay policy) is committed before `execute()`. + - `replay: "safe"` → the call reruns on recovery. + - default `unsafe` → it does **not** rerun; the model receives a `pi.tool-result` with diagnostic code `interrupted` and the durable partial output committed so far. + - This is a deliberate product decision, not a limitation to paper over. +- **Application state**: `runtime.memo(name, value)` is a first-writer-wins durable memo. A rerun sees the memo and can skip the effect. Memos disappear when the task becomes terminal. +- **Code Mode**: scripts run deterministically (virtual clock, seeded randomness, UTC) and every tool result is journaled before delivery. On restart the script reruns and receives journaled results in order; a changed path, sandbox fingerprint, or expired deadline fails the call instead. +- **Compaction**: durable task; a manual/background summary is placed at a turn boundary; a summary that would cut before the current context settles `stale`. +- **Storage errors**: fatal to the Session. Nothing after the error publishes; reopen to recover. Custom storage backends are responsible for retrying their own transient failures. + +### Storage and deployment + +Shipped backends (`README` §Storage): + +- **Memory** — tests only. +- **SQLite** (Node `node:sqlite`) — WAL, `synchronous = NORMAL`: commits survive process crashes; the newest commit may be lost on power/host failure. +- **JSONL** — append-only file per record, optional `fsync`; keeps the whole history in memory too, so use SQLite for long-running agents. +- **Cloudflare Durable Object SQLite** — commits are the DO's storage transactions. Main's README adds `onWake`/`wake()` so the DO arms its own alarm and hibernates while tasks sleep (unreleased in 1.3.0). + +Constraints: + +- **One process owns a storage at a time.** No cross-process locking. Other clients attach to that process (views/watches over an application-chosen transport). +- Node `>= 22.19`, ESM, TypeScript types shipped. +- Portable SQLite/JSONL cores avoid Node APIs so they run on Bun or in Workers with a small adapter. + +Observed SQLite schema in a 1.3.0 test database (9 tables): `conversations`, `entries`, `tasks`, `submissions`, `documents`, `document_revisions`, `record_ids`, `durable_metadata`, `durable_schema`. Entries and task records are JSON in `TEXT` columns with `json_valid` checks; indexes cover conversation/kind/status lookups and document addresses. This is a compact, inspectable store — a transcript can be reconstructed with plain SQL. + +### Extensions, tools, environment + +- **Registry**: extensions are named bundles of tools, prompt sections, hooks, wrappers, and tasks. Conversations store extension/tool **names**, never code, so they survive restarts and pick up whatever the new process installs. Registry changes are live; an in-flight call finishes on the code it started with. +- **Tools**: TypeBox-validated arguments; three result channels — `output` (model), `structuredOutput` (programs/nested calls), `details` (UI). Nested tool calls (`api.executeTool`) are their own tasks with a key for replay reattachment. +- **Code Mode**: model writes JavaScript in a QuickJS sandbox; tool calls become nested durable calls; journaled replay as above. +- **Environment**: `env({ cwd, conversationId })` builds an `ExecutionEnv` per tool call. Built-in `NodeExecutionEnv` gives full host FS/shell. A remote/container env can put tools on another machine. **No sandbox is included by default** — isolation is entirely the env implementation's job. +- **Observability**: `pi.durable.*` OpenTelemetry-style spans/events/metrics with content-free attributes (IDs and control metadata only, never messages/arguments/results). `watch()`, `viewState()`, and `watchEvents()` expose committed state for multi-client UIs. + +## Empirical checks (run 2026-10-20) + +Environment: Node 22.22.2, fresh install of `pi-durable@1.3.0` + `pi-ai@1.3.0` + `chord@1.3.0` (91 packages, 111 MB `node_modules`). + +1. **Basic chat via faux provider** — worked as documented: submit → `pi.generation` → `pi.system` + `pi.assistant` entries; transcript `['pi.user', 'pi.system', 'pi.assistant']`. +2. **Crash mid-phase, memo vs no memo** — custom task, `SIGKILL` after committing a memo but before committing the outcome; reopen with `resume()` + `waitForIdle()`: + - with `runtime.memo("effect", true)`: rerun logged `EFFECT-SKIPPED-BY-MEMO`, task completed; effect ran exactly once total. + - without memo: rerun logged `EFFECT-RAN` again; effect ran twice total. **Confirmed at-least-once phase execution.** +3. **Crash mid-tool-call** — faux model issues a tool call; tool `SIGKILL`s the process during `execute()`: + - `replay: "unsafe"` (default): on reopen the tool did **not** rerun; the transcript got a `pi.tool-result` with `diagnostics: [{ severity: "error", code: "interrupted", message: "Tool slow was interrupted and may have partially run" }]`; generation continued with the next model turn. + - `replay: "safe"`: the tool reran (`TOOL-RAN` printed again) and completed; generation continued. +4. **Cold-open wait race in 1.3.0** — after `SIGKILL`, calling `harness.waitForTask(id)` immediately after `Harness.open()` returned the stored **pending** record instead of waiting (its type says `SettledTask`, i.e. terminal). `resume()` + `waitForIdle()` + then `waitForTask()` returned the terminal outcome. Likely a reconciliation race (`#live` not populated when the wait registers). Observed behavior, reported here as an implementation caveat; needs upstream confirmation before calling it a bug. +5. **Version gap confirmed** — installed 1.3.0 contains no `sleepUntil`, `onWake`, `wake()`, `restartLimit`, or `sleeping` status, although the README on `main` documents them. The 1.3.0 SQLite `tasks` table check constraint only allows `pending/running/waiting/completing/terminal`. **Do not build against main's README thinking it is the published API.** + +## Claims vs. guarantees + +- **"Committed before anything is shown"** is directionally true but not literal: streamed partials are committed every ~100 ms, and the spec's own footgun list says a crash may lose the current uncommitted throttle window. Committed state is durable; the last 100 ms of visible tokens may not be. +- **"Pick up where it stopped"** means *from the last committed phase boundary*, not from an arbitrary instruction. A phase reruns from its top; side effects after the checkpoint repeat unless memoized/idempotent. +- **"Exactly-once"** applies to **submission admission** (`requestId`), not to tool effects, model calls, or webhooks. There is no outbox/transactional-messaging layer. +- **Failure detection is fast**: a storage error fails the whole Session; code errors fail only their unit (task phase faults the task, tool throws → error result, hook throws → reported and skipped, commit callback throws → rollback). +- **Crash loops**: `settings.restartLimit` (default 5) parks a task that crashes 5 times without a new checkpoint — added on main, not in 1.3.0. This is a necessary guard for deterministic crashes; without it, a poison task resumes forever. + +## Non-goals (spec §13) + +No whole-Session DOM; no visible-undurable publication; no independent event journal; no session-scoped rewindable documents; no automatic checkpoint heuristic; no automatic third-party view mounting; no CRDT/offline multi-writer merge; no SQL translation of arbitrary Chord ops; no JSONL global compaction or corruption repair; no forced termination of non-cooperative extension code inside one process. + +## Maturity and adoption + +- `pi-durable` 1.0.0 shipped **2026-10-01**; latest published **1.3.0 on 2026-10-15**. 16 npm versions in ~4 weeks; `[Unreleased]` already contains breaking changes (removal of `TaskRuntime.sleep()`, schema v2 migration that cannot roll back to 1.3.0). +- npm: **387,665 downloads in the week ending 2026-10-18**; 740,046 since first publish (2026-09-19). For context, `pi-ai` did 4.65 M that week and `chord` 1.91 M. +- GitHub: the `pi` monorepo has **116,619 stars / 14,877 forks** (2026-10-20), MIT. `packages/durable` has 175 path commits as of that date, the oldest returned by the API from 2026-09-18 (the package was split out of an earlier "Pico" package); 1.0.0 released 2026-10-01. +- Vendor claim: ~15,000 lines of source excluding tests (~150k GPT tokens / ~250k Claude tokens); storage backends ~3,000 lines. That was accurate at launch: we count **17,662 raw lines / 16,412 non-blank** across 60 `src/**/*.ts` files at tag `v1.0.0`. On `main` at 2026-10-20 it is **26,028 raw lines / 24,342 non-blank** across 84 files — roughly +47% in 19 days. +- Language: TypeScript only. The launch FAQ says a Rust port is "very easy" and "not ruled out", but not planned — relevant to our Rust-core preference. + +## Why this matters for the taxonomy + +pi-durable demonstrates that **durability can be a property of the harness library**, not only of the substrate. celld/rivet answer "where does state live and how does compute wake?"; pi-durable answers "what must the loop commit, in what order, and what happens on reopen?" The two are composable: the package's natural production shape is one Session per Cloudflare Durable Object (or one Node process) with SQLite as the commit log. + +It also imports durable-execution vocabulary into agent infrastructure without pretending to be Temporal: + +- effect sandwich ≈ activity with a durable intent record +- memo ≈ idempotency key with storage +- submission `requestId` ≈ client-supplied idempotency key +- code-mode journal ≈ deterministic replay of a workflow +- task ownership tree ≈ structured concurrency / cancellation scopes + +## Implications for our stack (hermes-vm) + +What to copy into Hermes-style tooling, regardless of language: + +1. **Commit-before-publish + `requestId` at every API boundary.** Kanban/webhook/task admission should be idempotent by a caller-supplied key; if our webhook path can be retried without one, it can duplicate work — worth verifying against the current handler. +2. **Effect sandwich for tools.** Separate "intent committed" from "effect performed" from "outcome committed"; make destructive tools (`bash`, deploy, send-message) `unsafe` by default and surface `interrupted` to the model instead of retrying. +3. **Task ownership tree with bottom-up abort.** The semantic we'd want behind `delegate_task`: a parent abort should cascade to children, and a parent should not report done while children run. +4. **Crash-loop guard.** A deterministic crash must not resume forever; park after N restarts and expose `retryTask`. +5. **Content-free telemetry.** IDs and control metadata only — a good default for our own traces. +6. **Compaction as a durable task**, not an in-memory step that can be lost mid-summary. + +What not to adopt: another runtime. hermes-vm already is the durable host (systemd, SQLite, kanban, session DBs); pi-durable solves a problem we have only if we build a *multi-client, long-running agent service* in TypeScript. If we do, the package itself (not a reimplementation) is the fastest path: SQLite on the VM, `NodeExecutionEnv`, and the `CodingTools`/Code Mode extensions. + +If we want the semantics in a Rust core + Python adapters, pi-durable is a strong **reference design** but not a dependency: the spec is the deliverable, and the storage contract (9 tables, atomic commit, JSON records) is small enough to reimplement. + +## Open questions + +- No Postgres/KV adapter shipped; the vendor says the interface is small, but no reference implementation exists. A multi-node deployment would need one plus a leader/lease model — explicitly out of scope today. +- Multi-client transport is application-owned: `watch()` gives committed operations, not a wire protocol. How much work is a real Slack/Web UI adapter? +- Security: the default Node env is full host access. There is no permission model, no per-tool sandbox, no policy engine beyond hooks. How does this compare with hypeman/E2B for untrusted tools? +- Cost control: model requests are resent after interruption; no token-level resume. How much does this cost at 100k-token contexts? No published benchmarks for storage or recovery latency (bench scripts exist: `bench:storage`, `bench:scheduler-fanout`, `bench:document-append`, etc.). +- Does `restartLimit`/`sleeping`/`onWake` ship in 1.4, and does the DO path become production-grade? +- Is the source size workable for a reimplementation? At `main` it is 26k raw lines, but much of that is TypeScript types and exhaustive JSDoc; the storage/task core is materially smaller. Worth a focused read of `src/harness/scheduler.ts` (108 KB) before committing to "small enough to reimplement". + +## Sources + +Primary (code/spec, highest trust): + +- [`packages/durable/README.md`](https://github.com/earendil-works/pi/blob/main/packages/durable/README.md) — user-facing API, storage, errors, examples. Accessed 2026-10-20. +- [`packages/durable/docs/spec.md`](https://github.com/earendil-works/pi/blob/main/packages/durable/docs/spec.md) — normative specification: terms/invariants (§1), records (§2), documents (§3), transactions (§4), tasks/effect sandwich/structured concurrency/sleep (§5), submissions (§6), extensions/tools (§7), built-in tasks (§8), views (§9), storage contract (§10), backends (§11), footguns (§12), non-goals (§13). +- [`packages/durable/CHANGELOG.md`](https://github.com/earendil-works/pi/blob/main/packages/durable/CHANGELOG.md) — release history; 1.0.0 2026-10-01 → 1.3.0 2026-10-15; unreleased breaking changes. +- [`packages/durable/package.json`](https://github.com/earendil-works/pi/blob/main/packages/durable/package.json) — version 1.3.0, MIT, Node >= 22.19, deps (pi-ai, chord, pi-codemode, pi-telemetry, TypeBox, Photon, diff). +- [`packages/chord/README.md`](https://github.com/earendil-works/pi/blob/main/packages/chord/README.md) — the document-state substrate: objects, references, observable JSON, deltas, hot reload, SQLite adapters. +- Source tree inspection: `src/harness` (scheduler 108 KB, tool 41 KB, generation 34 KB), `src/session` (transaction 42 KB), `src/storage` (memory 33 KB, sqlite, jsonl), `src/codemode`, `src/tools`, `src/env`. +- GitHub API: repo metadata (116,619 stars, MIT) and commit history for `packages/durable` (175 path commits, oldest 2026-09-18; package split out of "Pico"), accessed 2026-10-20. +- npm registry API: `@earendil-works/pi-durable` dist-tags (latest 1.3.0), first publish 2026-09-19, 16 versions; downloads last week 387,665. + +Vendor/community (reported, not independently verified): + +- [Earendil Engineering, "Pi Durable" (2026-10-01)](https://earendil.com/posts/pi-durable/) — launch post: positioning ("harness that runs anywhere"), architecture tour, multi-client/multiplayer claims, ~15k LOC claim, FAQ on TypeScript/Rust. +- [explainx.ai, "Pi 1.0 + Durable" (Oct 2026)](https://www.explainx.ai/blog/earendil-pi-1-durable-minimal-harness-2026) — secondary coverage; useful only for reception framing. +- [MindStudio, "Pi Durable: What It Is and Why It Matters"](https://www.mindstudio.ai/blog/pi-durable-long-running-agents) — secondary summary; not used for factual claims. + +Verified by execution in this research (reproducible): + +- Install `@earendil-works/pi-durable@1.3.0` with Node 22.22.2; run faux-provider chat; kill -9 mid-phase with and without `runtime.memo`; kill -9 inside `replay: "safe"` and default `unsafe` tool calls; inspect SQLite schema. Scripts and DBs under `/tmp/pi-durable-research/test/` on the research host; results summarized in "Empirical checks" above. diff --git a/durable-agent-runtime/sources.md b/durable-agent-runtime/sources.md index d53b921..42b62c1 100644 --- a/durable-agent-runtime/sources.md +++ b/durable-agent-runtime/sources.md @@ -24,6 +24,21 @@ Full reference list for the research topic. One line per source: what it is, why - [e2b-dev/infra (GitHub)](https://github.com/e2b-dev/infra) — infrastructure powering E2B Cloud: Firecracker-based sandbox VMs for AI agents, template system, snapshot cold starts (sub-30ms). Open source; the substrate Amp orbs run on (per Amp's security reference). Also E2B docs: [e2b.dev](https://e2b.dev/docs). - [Spheron guide: E2B vs Daytona vs Firecracker (2026)](https://www.spheron.network/blog/ai-agent-code-execution-sandbox-e2b-daytona-firecracker/) — sandbox architecture comparison: E2B managed (CPU-only, snapshot cold starts), Daytona, Firecracker. Snapshot-restore + sandbox pooling math for cold-start latency. +## Durable harness library — pi-durable + +Researched 2026-10-20 against `main` (`48c8f32`) and published `1.3.0`; full write-up in [[durable-agent-runtime/pi-durable|pi-durable]]. + +- [packages/durable README](https://github.com/earendil-works/pi/blob/main/packages/durable/README.md) — API tour: concepts, persist/resume, extensions, tools, code mode, settings, environment, compaction, child tasks, storage, errors. Documents some unreleased features (durable sleeps, `onWake`, crash-loop guard) not present in 1.3.0. +- [packages/durable docs/spec.md](https://github.com/earendil-works/pi/blob/main/packages/durable/docs/spec.md) — normative spec (~5.6k lines). Key sections: §1 invariants, §5.2 effect sandwich, §5.5 structured concurrency, §8.3 generation recovery, §8.4 tool replay policy, §12 API footguns, §13 non-goals. +- [packages/durable CHANGELOG](https://github.com/earendil-works/pi/blob/main/packages/durable/CHANGELOG.md) — 1.0.0 (2026-10-01) through 1.3.0 (2026-10-15); Unreleased has breaking changes (removes `TaskRuntime.sleep()`, SQLite schema v2 with no rollback). +- [packages/durable package.json](https://github.com/earendil-works/pi/blob/main/packages/durable/package.json) — v1.3.0, MIT, Node >= 22.19; deps: pi-ai, chord, pi-codemode, pi-telemetry, TypeBox, Photon, diff. +- [packages/chord README](https://github.com/earendil-works/pi/blob/main/packages/chord/README.md) — document-state substrate: objects at addresses, observable JSON, delta tracking, hot reload, SQLite adapters. +- [Earendil, "Pi Durable" launch post (2026-10-01)](https://earendil.com/posts/pi-durable/) — vendor positioning and architecture tour; claims ~15k LOC excluding tests, "runs anywhere there is a JavaScript runtime", multi-client/multiplayer design; FAQ says a Rust port is possible but not planned. +- [npm: @earendil-works/pi-durable](https://www.npmjs.com/package/@earendil-works/pi-durable) — first publish 2026-09-19; 16 versions; 1.3.0 latest as of 2026-10-20; npm API reports 387,665 downloads for the week ending 2026-10-18. +- [GitHub: earendil-works/pi](https://github.com/earendil-works/pi) — monorepo metadata at 2026-10-20: 116,619 stars, 14,877 forks, MIT; `packages/durable` has 175 path commits (oldest API result 2026-09-18), 1.0.0 released 2026-10-01. + +Empirical (this research, 2026-10-20): installed 1.3.0 under Node 22.22.2; ran faux-provider chat, `SIGKILL` mid-task with/without `runtime.memo`, `SIGKILL` mid-tool for `replay: "safe"` vs default `unsafe`, and dumped the SQLite schema. Results and caveats (including a cold-open `waitForTask` race) are in [[durable-agent-runtime/pi-durable|pi-durable]]. + ## Harness engineering / agent-as-library - [gakonst/nanocodex (GitHub)](https://github.com/gakonst/nanocodex) — library-first Rust reimplementation of the Codex harness ("blazing-fast, minimal, library-first"). Typed turns/tool registry/event streams/steering/cancellation/forks over the OpenAI Responses WebSocket API. Agent session lives in your process; no app server or durable control plane. Crates: facade, agent lifecycle, typed OpenAI boundary, tools (shell/Code Mode/MCP), observability (OTel), experimental voice + VM-backed tools. Thin CLI/TUI, Python, Node/WASM, React consumers. Auth shares `~/.codex/auth.json`.