diff --git a/durable-agent-runtime/analysis.md b/durable-agent-runtime/analysis.md new file mode 100644 index 0000000..b7e653b --- /dev/null +++ b/durable-agent-runtime/analysis.md @@ -0,0 +1,58 @@ +# Analysis — Durable Agent Runtimes & Harness Engineering + +> Part of [[durable-agent-runtime/index|Durable Agent Runtimes & Harness Engineering]] + +## The four primitive families + +| Family | Example | Compute primitive | Where state lives | Cold start | Hibernation | +| --- | --- | --- | --- | --- | --- | +| Durable objects | celld (Deno) | V8 isolate running Worker bundles | Per-object SQLite, replicated to S3 | Fast (isolate) | Idle cells hibernate | +| Actors | rivet | Lightweight process (Node/Bun/Deno) | In-memory + SQLite/BYO DB | ~20ms | Actors hibernate when idle | +| VM sandboxes | hypeman, E2B | Firecracker/Cloud Hypervisor/QEMU microVM | VM disk (snapshot for standby) | ~30s VM / sub-30ms from snapshot | standby = snapshot to disk | +| Harness libraries | nanocodex, Codex, pi | In-process agent loop (WebSocket to LLM) | In-process session / thread store | Instant | N/A (client-side) | + +The industry convergence: **compute is disposable, state is durable elsewhere.** + +- Amp orbs: ephemeral E2B sandbox + durable thread on Amp Server (PostgreSQL). Sandbox destroyed on archive; conversation survives. +- celld: nodes are replaceable; the S3 bucket is the durable source of truth. Object = its own SQLite DB → sharding and blast-radius containment by construction, no consensus. +- rivet: actor state in-memory for 0ms reads, persisted for durability; runs indefinitely, sleeps when idle. +- hypeman: standby = snapshot VM to disk, restore in milliseconds — VM-level hibernation. + +## The two hard problems everyone solves + +1. **Cold start latency** — the gap between "VM ~30s" and "actor ~20ms" is closed with snapshots (E2B, hypeman standby, celld/rivet hibernation) or pre-provisioned pools. This is why orbs can feel local despite being remote. +2. **State checkpointing & recovery** — thread JSONL (Amp), SQLite replication (celld), actor state persistence (rivet), VM disk snapshots (hypeman). Everything is replayable; nothing is lost on compute failure. + +## Tool interaction: code as the tool interface + +Three independent sources converge on the same insight: + +- **Cloudflare Code Mode**: present MCP tools as a generated TypeScript API and let the LLM write code that calls it. LLMs are trained on billions of lines of real code but only contrived tool-call token sequences. Writing code also skips feeding every intermediate tool result back through the context window. +- **nanocodex**: typed `Turn`/`TurnResult` values + event stream over WebSocket instead of stdout parsing. "stdout parsing for agent loops breaks constantly in edge cases; typed events give reliable state checkpoints and clean prompt cache hits." +- **Prime Agent RLM**: the model's only tool is a persistent IPython REPL; context is a variable, subagents are function calls. + +The unifying pattern: **the harness should look like a programming environment, not a tool-call menu.** This matters directly for how we build agent environments on hermes-vm (and why pi's minimalism / 4-tool philosophy won). + +## What this means for hermes-vm (our stack) + +Already have (the "durable compute host" layer): +- Persistent remote dev VM (hermes-vm), systemd services, kanban DB, session DBs, herdr panes, tailscale. +- herdr = the "shared tmux + durable agent loop" half of an orb, already built. + +Genuine gaps (the "environment contract" layer): +- Per-repo `.agents/setup`-style bootstrap + AGENTS.md per repo — cheap, ~90% of the reliability win. +- Idempotent dev-server script + ports metadata file + `/__dev`-style auth bypass for our apps. +- Event-driven wake: Forgejo webhooks → kanban task (self-hosted equivalent of Amp's event-driven orbs). +- Formalize tailscale serve as per-repo portals. + +Not worth building on a single box: +- A distributed durable-object layer (celld) or actor platform (rivet) — solves multi-node, multi-region, scale-to-zero billing problems we don't have. One VM = one durable host. +- An E2B-style per-thread VM fleet — same reason. NixOS already gives cheaper isolation: `nix develop`, rootless podman (already enabled via `modules/dev/containers.nix`), worktrees, NixOS containers/systemd-nspawn if stronger isolation is ever needed. + +Interesting but optional: +- **hypeman** — if we ever want real per-task VM isolation with snapshot/standby on our own box, hypeman is the lean OSS option (multi-hypervisor, OCI images, ms restore). Watch it; don't adopt yet. +- **Cloudflare Code Mode pattern** — worth revisiting for our Effect TS stack: present our MCP/REST tools as a typed API and let agents write code against them, rather than exposing raw tool-call schemas. + +## Verdict + +Build the **environment contracts** (setup hooks, AGENTS.md, dev-server skills, event wake) on the existing durable host. Do not build the sandbox platform or a distributed state layer. herdr + systemd + kanban + tailscale already provide the durability; the missing piece is making the environment "assume an agent and tell it where the light switches are." diff --git a/durable-agent-runtime/index.md b/durable-agent-runtime/index.md new file mode 100644 index 0000000..193a223 --- /dev/null +++ b/durable-agent-runtime/index.md @@ -0,0 +1,72 @@ +--- +title: Durable Agent Runtimes & Harness Engineering +description: Survey of durable, stateful agent execution primitives — durable objects (celld), actors (rivet), multi-hypervisor VM runtimes (hypeman, E2B/Amp Orbs), harness-as-library (nanocodex), code-as-tool-calling (Cloudflare Code Mode) — and what they mean for building a self-hosted orb-like runtime on hermes-vm. +status: active +category: technical +tags: [durable-objects, actors, agent-runtime, agent-harness, sandbox, microvm, self-hosting, orbs, stateful, durable-execution] +draft: true +created: 2026-08-07 +updated: 2026-08-07 +origin: buzz://d8a718be-031f-4a6e-9f5c-a55466641654/4123d35532d0b2cbfb80a0a34f79fb3e4e63c16c25b77d43bc0f10bb178e4f1b +--- + +# Durable Agent Runtimes & Harness Engineering + +## Summary + +Research into how modern agent execution environments achieve **durability and statefulness** — where agent state lives, what compute primitive runs the agent, and how the environment hibernates and wakes. Covers the spectrum from in-process harness libraries (nanocodex) and V8 isolates (celld, Cloudflare Workers) to actor runtimes (rivet) and microVM sandboxes (hypeman, E2B/Amp Orbs). Goal: derive a build plan for a self-hosted, orb-like runtime on our NixOS VM (hermes-vm) without overbuilding. + +## Research Question + +What are the architectural primitives for building a durable, stateful agent runtime (sandboxed or not), and which combination fits a single self-hosted NixOS dev VM? + +## Scope + +- Included: durable objects / durable execution (celld, rivet, Durable Objects), agent harness engineering (nanocodex, Amp orbs, Prime Agent RLM), sandbox runtimes (hypeman, E2B, Cloudflare isolates), tool-calling vs code-calling (Cloudflare Code Mode) +- Excluded: model training/inference infra, prompt engineering, RLHF +- Sufficient answer: a taxonomy of primitives + a recommendation for what to build (and what NOT to build) on hermes-vm + +## Key Findings + +- **The 2026 pattern is unanimous**: agent compute is disposable, agent state is durable elsewhere. Amp: sandbox is ephemeral, the thread (prompts, tool calls, results) lives on the server. celld: nodes are replaceable, the S3 bucket is the source of truth. Rivet: actor state persists while the process may hibernate. +- **Four primitive families emerged**: + 1. Durable object/execution stores — celld (each object = its own SQLite DB, S3 replication, no consensus) + 2. Actor runtimes — rivet (in-memory state + persistence, ~20ms cold start, scale to zero, queues/workflows/scheduling) + 3. Sandbox/VM runtimes — hypeman (multi-hypervisor OCI-in-VM), E2B (Firecracker microVMs, snapshot restore, used as Amp's orb substrate) + 4. Harness libraries / agent loops — nanocodex (Rust, agent as in-process library, typed events over WebSocket), Codex, pi, Prime Agent RLM +- **Scale-to-zero / hibernation is universal**: Amp orbs pause after 5 min idle; celld cells hibernate; rivet actors hibernate when idle; hypeman standby = snapshot VM to disk, resume in milliseconds; E2B snapshot reuse gives sub-30ms cold starts. +- **Tool interaction is converging on "code as the tool interface"**: Cloudflare Code Mode converts MCP tools into a TypeScript API and has the LLM write code that calls it (LLMs are far better at code than at synthetic tool-call tokens). nanocodex's thesis is the same: typed events beat stdout parsing. Prime Agent's RLM: a persistent IPython REPL is the model's only tool. pi's minimalism: 4 tools, no MCP bloat. +- **Two hard problems everyone solves**: (a) cold start latency (actor ~20ms vs VM ~30s, mitigated by snapshot/pooling), (b) state checkpointing and recovery (SQLite replication, thread JSONL, VM snapshots, actor state persistence). +- **For our stack**: hermes-vm is already a durable compute host (systemd services, kanban DB, session DBs, herdr panes). The real gap is a per-task disposable environment contract + event wake, not a distributed state layer. celld/rivet solve problems we don't have on a single box; hypeman is interesting for per-task VMs but likely overkill. + +## Sources + +For the full reference list see [[durable-agent-runtime/sources|Sources]]. + +- [[durable-agent-runtime/sources|Sources]] — all links, one-line descriptions + +## Detail Files + +- [[durable-agent-runtime/sources|Sources]] — full reference list +- [[durable-agent-runtime/analysis|Analysis]] — taxonomy, comparison matrix, implications for our build +- [[durable-agent-runtime/next-steps|Next Steps]] — follow-up questions and build plan + +## Related Research + +- Amp Orbs research note (workspace): `RESEARCH/AMP_ORBS_REMOTE_DEV.md` +- Prime Agent / Continual Harness (arXiv 2605.09998) — self-improving harness state +- Forgejo AI reviewer harness (workspace skill reference) — where nanocodex was first evaluated + +## Discussion + +Triggered from the Amp Orbs thread (2026-08-07). User connected the dots: orbs, nanocodex, durable objects (celld), actors (rivet), multi-hypervisor VM runtimes (hypeman), and Cloudflare Code Mode all belong to one research area — "durable object, stateful machine, harness engineering either sandboxed or not." Asked for centralized structured research he can digest, with the option of parallel Luna delegation. Completed directly (all primary sources already extracted; delegation channel has known instability). + +## Next Steps + +- [ ] Decide Phase 1 scope: per-repo `.agents/setup`-style contracts + AGENTS.md per repo on hermes-vm +- [ ] Evaluate hypeman as an opt-in per-task VM runtime (snapshot/standby) vs NixOS containers +- [ ] Revisit Code Mode pattern for our Effect TS stack (tools as typed API, not tool-calling) + +## Conclusion + +The research area is **durable, stateful agent runtime and harness engineering — sandboxed or not**. Four primitive families cover the space; for a single self-hosted NixOS VM the right move is environment contracts + event wake on the existing durable host, not building a distributed durable-object layer. diff --git a/durable-agent-runtime/next-steps.md b/durable-agent-runtime/next-steps.md new file mode 100644 index 0000000..a7704bd --- /dev/null +++ b/durable-agent-runtime/next-steps.md @@ -0,0 +1,18 @@ +# Next Steps — Durable Agent Runtimes & Harness Engineering + +> Part of [[durable-agent-runtime/index|Durable Agent Runtimes & Harness Engineering]] + +## Open questions + +- [ ] Which repos get the `.agents/setup`-style contract first (clan-private, law retrieval, idx-cli)? +- [ ] Should the dev-server + ports-file pattern be a shared script library in clan-private or per-repo? +- [ ] Is hypeman worth an evaluation build on hermes-vm (per-task VM with standby) vs NixOS containers? +- [ ] Revisit Cloudflare Code Mode for our Effect TS stack: tools-as-typed-API vs tool-calling. +- [ ] Does the Forgejo webhook → kanban wake need a new service, or does an existing cron/webhook path cover it? + +## Decisions made (2026-08-07) + +- Research topic: **durable-agent-runtime** — covers durable objects, stateful machines, and harness engineering (sandboxed or not). +- Do NOT build a distributed durable-object layer (celld) or actor platform (rivet) on a single box. +- Do NOT build an E2B-style per-thread VM fleet; use NixOS-native isolation when needed. +- DO build environment contracts: per-repo setup hooks, AGENTS.md per repo, dev-server skills, event wake. diff --git a/durable-agent-runtime/sources.md b/durable-agent-runtime/sources.md new file mode 100644 index 0000000..d53b921 --- /dev/null +++ b/durable-agent-runtime/sources.md @@ -0,0 +1,40 @@ +# Sources — Durable Agent Runtimes & Harness Engineering + +> Part of [[durable-agent-runtime/index|Durable Agent Runtimes & Harness Engineering]] + +Full reference list for the research topic. One line per source: what it is, why it matters. + +## Amp Orbs (remote sandbox + durable agent loop) + +- [Agents in Orbs (Amp news, 2026-06-30)](https://ampcode.com/news/agents-in-orbs) — launch announcement. Orbs = machines where agents run without supervision; fresh orb per thread with code/plugins/tools; 32GB/16-core default, $1.32/hr, billed by the minute, auto-pause after 5 min idle. +- [Putting an Agent in an Orb (Amp note, 2026-07-02)](https://ampcode.com/notes/putting-an-agent-in-an-orb) — the deep-dive: `.agents/setup` + `.agents/resume` repo hooks, snapshot reuse up to 24h, dev-server skill (`ensure-dev-server.sh`), `/__dev` auth endpoints, `.amp/dev-ports.json`, `.amp/in/` logs, 41 AGENTS.md files. +- [What I Want to Tell You About Orbs (Amp note, 2026-08-04)](https://ampcode.com/notes/what-i-want-to-tell-you-about-orbs) — philosophy/experience piece. "Ingredients": secure sandbox, scale to zero, ephemeral, durable agent loop, controllable from web/phone/desktop, portals, multiplayer, automations. Evidence-first workflow: agents must produce proof (screenshots, videos). +- [Orbs manual (Amp docs)](https://ampcode.com/manual/orbs) — full spec: orb sizes, billing, `.agents/setup`/`.agents/resume`/`.amp/services.yaml`/portals, webhooks (event-driven orbs), OIDC workload identity (`amp orb id-token`), secrets hierarchy (personal > project > workspace). +- [Amp Security Reference](https://ampcode.com/security) — the architecture disclosure: Amp Client (CLI) + Amp Server (GCP multi-tenant, PostgreSQL thread storage) + **e2b** for sandbox compute; providers (Anthropic/OpenAI/xAI/Meta/Gemini/Bedrock/Baseten/Fireworks for inference, WorkOS auth, Stripe billing). No self-hosted deployment offered. + +## Durable objects / durable execution + +- [denoland/celld (GitHub)](https://github.com/denoland/celld) — self-hosted, distributed Durable Objects. Each object is its own SQLite DB, replicated to an S3-compatible bucket; nodes coordinate through the bucket with no control plane/consensus. V8-embedded, runs Wrangler bundles. Idle cells hibernate. By Deno Land (ry@deno.com). ~222★, Apache-2.0. +- [rivet-dev/rivet (GitHub)](https://github.com/rivet-dev/rivet) — Rivet Actors: long-running lightweight processes for stateful workloads. In-memory state + automatic persistence (SQLite/BYO DB), ~20ms cold start, ~0.6KB memory/instance, scale to zero, WebSockets/queues/workflows/scheduling, global edge. Self-host (single Rust binary) or cloud. ~5.8k★, Apache-2.0. Engine sub-components: Pegboard (orchestrator), Gasoline (durable execution), Guard (traffic proxy), Epoxy (multi-region KV, EPaxos). +- [celld.dev docs](https://celld.dev/docs) — official docs for celld (deployment, limitations, security pages). + +## Sandbox / VM runtimes + +- [kernel/hypeman (GitHub)](https://github.com/kernel/hypeman) — multi-hypervisor VM runtime for OCI images: Cloud Hypervisor, Firecracker, QEMU on Linux; Apple Virtualization.framework on macOS. Docker-compatible CLI (`run`, `exec`, `ps`, `logs`), standby/restore (snapshot VM to disk, resume in ms), built-in ingress/TLS/subdomain routing, GPU passthrough, remote JWT-authenticated API. ~303★, MIT. +- [e2b-dev/infra (GitHub)](https://github.com/e2b-dev/infra) — infrastructure powering E2B Cloud: Firecracker-based sandbox VMs for AI agents, template system, snapshot cold starts (sub-30ms). Open source; the substrate Amp orbs run on (per Amp's security reference). Also E2B docs: [e2b.dev](https://e2b.dev/docs). +- [Spheron guide: E2B vs Daytona vs Firecracker (2026)](https://www.spheron.network/blog/ai-agent-code-execution-sandbox-e2b-daytona-firecracker/) — sandbox architecture comparison: E2B managed (CPU-only, snapshot cold starts), Daytona, Firecracker. Snapshot-restore + sandbox pooling math for cold-start latency. + +## Harness engineering / agent-as-library + +- [gakonst/nanocodex (GitHub)](https://github.com/gakonst/nanocodex) — library-first Rust reimplementation of the Codex harness ("blazing-fast, minimal, library-first"). Typed turns/tool registry/event streams/steering/cancellation/forks over the OpenAI Responses WebSocket API. Agent session lives in your process; no app server or durable control plane. Crates: facade, agent lifecycle, typed OpenAI boundary, tools (shell/Code Mode/MCP), observability (OTel), experimental voice + VM-backed tools. Thin CLI/TUI, Python, Node/WASM, React consumers. Auth shares `~/.codex/auth.json`. +- [tact (github.com/clabby/tact)](https://github.com/clabby/tact) — a terminal interface for nanocodex, launched Jul 2026; browser-based `/review` interface (agent-authored visual code review). Proof of the build-your-own-agent-on-nanocodex loop. +- [Cloudflare: Code Mode — the better way to use MCP (blog, 2025-09-26)](https://blog.cloudflare.com/code-mode/) — converts MCP tool schemas into a TypeScript API and asks the LLM to write code that calls the API, instead of presenting tools directly. Run in a sandboxed V8 isolate via the Worker Loader API; no containers. Finding: LLMs are much better at writing code than at synthetic tool-call tokens; avoids copying intermediate tool results back through the context. +- [Cloudflare Agents SDK codemode docs](https://github.com/cloudflare/agents/blob/main/docs/codemode.md) — the codemode helper + worker loader usage. +- [Prime Intellect: Recursive Language Models (blog, 2026-01-01)](https://www.primeintellect.ai/blog/rlm) — RLM paradigm: persistent Python REPL as the model's only tool, context as a variable, sub-LLM calls as functions. Related arXiv: 2512.24601 (Recursive Language Models), 2605.09998 (Continual Harness: Online Adaptation for Self-Improving Foundation Agents). + +## Context from our own work + +- Buzz thread origin: `buzz://d8a718be-031f-4a6e-9f5c-a55466641654/91604ae52f7fc33d6e43543f3b7c355f65f353d4a6bc820b6dd8e8d0fd70f2ae` (Amp Orbs research) and the triggering event for this topic. +- Workspace note: `RESEARCH/AMP_ORBS_REMOTE_DEV.md` — Amp Orbs deep research with system-design follow-up (E2B substrate, lifecycle, mapping to our stack). +- Session research on nanocodex (2026-07-31): library-first Rust SDK, vs Codex/pi/OpenCode positioning, reviewer-harness relevance. +- clan-private: `machines/hermes-vm/configuration.nix` (hermes-vm host config), `modules/dev/containers.nix` (rootless podman enabled), `services/buzz/` (Buzz relay/gateway/agents), herdr (persistent PTY / tmux replacement).