Restructure agent research under agent-infra umbrella
- Add agent-infra umbrella topic (hub for state, harness, model-access, collaboration lenses) - Split harness-engineering out of durable-agent-runtime; fold in Cloudflare Code Mode finding - Narrow durable-agent-runtime to state/durability lens; retitle - Tag codex-app-server-byo-auth as agent-infra subtopic - Document umbrella-topic convention in README
This commit is contained in:
hermes
parent
579057e977
commit
c978db1437
8 changed files with 172 additions and 39 deletions
|
|
@ -28,6 +28,10 @@ Each research topic is a **directory** at the vault root with an `index.md` entr
|
|||
- **Big topics**: split into detail files as they grow. Link them from `index.md` with path-qualified wikilinks (e.g. `[[topic-slug/sources|Sources]]`) to avoid ambiguous basenames across topics.
|
||||
- Detail file names are up to you — `sources`, `analysis`, `methodology`, `notes`, `summary`, `next-steps` are common patterns.
|
||||
|
||||
### Umbrella topics
|
||||
|
||||
Some topics are **umbrella hubs** (e.g. `agent-infra`) whose job is composition: they frame a general research area and link to subtopic directories. Umbrellas are ordinary topic directories — keep the actual research in the subtopics, and tag subtopics with the umbrella's tag (e.g. `agent-infra`) for filtering.
|
||||
|
||||
### Frontmatter (in `index.md`)
|
||||
|
||||
```yaml
|
||||
|
|
|
|||
52
agent-infra/index.md
Normal file
52
agent-infra/index.md
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
---
|
||||
title: Agent Infrastructure
|
||||
description: Umbrella topic for the substrate layer agents run on — compute/sandbox, state/durability, harness/loop engineering, model access & auth, collaboration/identity. Subtopic hub: durable-agent-runtime, harness-engineering, codex-app-server-byo-auth, radicle-decentralized-git.
|
||||
status: active
|
||||
category: technical
|
||||
tags: [agent-infra, agent-runtime, harness, sandbox, durable-execution, byok, agent-protocols, self-hosting, orchestration]
|
||||
draft: true
|
||||
created: 2026-08-11
|
||||
updated: 2026-08-11
|
||||
origin: buzz://d8a718be-031f-4a6e-9f5c-a55466641654/c66f017974f748bd6db019c913d3a6f59de349e0cbd90ae38c222e9640521d63
|
||||
---
|
||||
|
||||
# Agent Infrastructure
|
||||
|
||||
## Summary
|
||||
|
||||
Umbrella topic for **agent infrastructure** — everything between the model and the user's product: what compute the agent runs on, where its state lives, how the agent loop is harnessed, how it gets model access, and how agents collaborate. Replaces the narrower "durable agent runtime" framing as the general home for this research area; each lens lives in a subtopic below.
|
||||
|
||||
## Subtopic Map
|
||||
|
||||
- [[durable-agent-runtime/index|Durable Agent Runtime]] — state & durability lens: durable objects (celld), actors (rivet), VM sandboxes (E2B, hypeman), scale-to-zero/hibernation, orbs
|
||||
- [[harness-engineering/index|Harness Engineering]] — agent loop & tool interface lens: nanocodex, pi, Codex protocols, code-as-tool-interface (Cloudflare Code Mode), skills/plugins
|
||||
- [[codex-app-server-byo-auth/index|Codex App Server & BYO Auth]] — model access & auth lens: embedding protocols, ChatGPT managed/external-token/API-key auth, BYOK harness pattern
|
||||
- [[radicle-decentralized-git/index|Decentralized Git (Radicle)]] — collaboration & identity lens: P2P forges, COBs as signed collaboration objects, agents as first-class participants
|
||||
|
||||
## Research Question
|
||||
|
||||
What does it take to run durable, stateful, sandboxed agents on our own infrastructure (hermes-vm), and where do we buy vs build?
|
||||
|
||||
## Key Findings
|
||||
|
||||
- **The 2026 convergence** (evidence in the subtopics): compute is disposable, state is durable elsewhere; and the harness should look like a programming environment, not a tool-call menu (code-as-tool-interface).
|
||||
- **For a single self-hosted box**: build environment contracts (setup hooks, AGENTS.md, event wake) on the existing durable host; don't build a distributed durable-object layer, actor platform, or VM fleet.
|
||||
- **Model access is a composable layer**: BYO-ChatGPT / BYOK is a model-access grant, not identity — one provider interface, multiple implementations (Effect Layer pattern).
|
||||
|
||||
## Detail Files
|
||||
|
||||
Subtopic index pages are the entry points; each carries its own analysis / sources / next-steps.
|
||||
|
||||
## Related Research
|
||||
|
||||
- Workspace notes: `RESEARCH/AMP_ORBS_REMOTE_DEV.md`, `RESEARCH/CODEX_APP_SERVER.md`
|
||||
- [[durable-agent-runtime/agent-collaboration|Agent-Native Collaboration]] — agents as first-class signed participants (Radicle COBs)
|
||||
|
||||
## Discussion
|
||||
|
||||
Restructured 2026-08-11 per user request: the main topic should be more general than "durable agent runtime" / "harness engineering". Created as umbrella hub; durable-agent-runtime narrowed to the state/durability lens, harness-engineering split out as its own topic (absorbing the Cloudflare Code Mode finding), existing topics tagged `agent-infra` and linked from here.
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [ ] Keep subtopics as the granular home for new research; add new subtopics (sandboxing, orchestration, identity) under this umbrella as they emerge
|
||||
- [ ] seo-playbook stays outside this umbrella (product/market lane)
|
||||
|
|
@ -3,7 +3,7 @@ title: Codex App Server — Embeddable Agent Protocol & BYO Auth
|
|||
description: Research into the Codex app-server JSON-RPC protocol — threads/turns/streaming, the three auth modes (ChatGPT managed, external tokens, API key), and the BYOK harness pattern for building no-sign-in apps where users bring their own model access (OpenRouter key or ChatGPT/Codex auth).
|
||||
status: active
|
||||
category: technical
|
||||
tags: [codex, app-server, byok, byo-auth, oauth, agent-protocol, json-rpc, harness, chatgpt, openrouter]
|
||||
tags: [codex, app-server, byok, byo-auth, oauth, agent-protocol, json-rpc, harness, chatgpt, openrouter, agent-infra]
|
||||
draft: true
|
||||
created: 2026-08-10
|
||||
updated: 2026-08-10
|
||||
|
|
@ -51,7 +51,8 @@ For the full reference list see [[codex-app-server-byo-auth/sources|Sources]].
|
|||
|
||||
## Related Research
|
||||
|
||||
- [[durable-agent-runtime/index|Durable Agent Runtimes & Harness Engineering]] — harness libraries family (nanocodex, Codex, pi); code-as-tool-interface pattern
|
||||
- [[agent-infra/index|Agent Infrastructure]] — umbrella topic
|
||||
- [[durable-agent-runtime/index|Durable Agent Runtime]] — harness libraries family (nanocodex, Codex, pi); code-as-tool-interface pattern
|
||||
- Workspace note: `RESEARCH/CODEX_APP_SERVER.md`
|
||||
|
||||
## Discussion
|
||||
|
|
|
|||
|
|
@ -1,17 +1,19 @@
|
|||
# Analysis — Durable Agent Runtimes & Harness Engineering
|
||||
# Analysis — Durable Agent Runtime
|
||||
|
||||
> Part of [[durable-agent-runtime/index|Durable Agent Runtimes & Harness Engineering]]
|
||||
> Part of [[durable-agent-runtime/index|Durable Agent Runtime]]
|
||||
> Harness/loop analysis moved to [[harness-engineering/index|Harness Engineering]] (2026-08-11).
|
||||
|
||||
## The four primitive families
|
||||
## The three primitive families
|
||||
|
||||
| Family | Example | Compute primitive | Where state lives | Cold start | Hibernation |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| Durable objects | celld (Deno) | V8 isolate running Worker bundles | Per-object SQLite, replicated to S3 | Fast (isolate) | Idle cells hibernate |
|
||||
| Actors | rivet | Lightweight process (Node/Bun/Deno) | In-memory + SQLite/BYO DB | ~20ms | Actors hibernate when idle |
|
||||
| VM sandboxes | hypeman, E2B | Firecracker/Cloud Hypervisor/QEMU microVM | VM disk (snapshot for standby) | ~30s VM / sub-30ms from snapshot | standby = snapshot to disk |
|
||||
| Harness libraries | nanocodex, Codex, pi | In-process agent loop (WebSocket to LLM) | In-process session / thread store | Instant | N/A (client-side) |
|
||||
| Collaborative substrate | Radicle COBs | Signed, CRDT-like Git commit DAGs | Git objects (replicated, self-certifying) | Instant (local git) | Repos persist; nodes sync on demand |
|
||||
|
||||
(Harness libraries — nanocodex, Codex, pi — moved to [[harness-engineering/index|Harness Engineering]].)
|
||||
|
||||
The industry convergence: **compute is disposable, state is durable elsewhere.**
|
||||
|
||||
- Amp orbs: ephemeral E2B sandbox + durable thread on Amp Server (PostgreSQL). Sandbox destroyed on archive; conversation survives.
|
||||
|
|
@ -24,16 +26,6 @@ The industry convergence: **compute is disposable, state is durable elsewhere.**
|
|||
1. **Cold start latency** — the gap between "VM ~30s" and "actor ~20ms" is closed with snapshots (E2B, hypeman standby, celld/rivet hibernation) or pre-provisioned pools. This is why orbs can feel local despite being remote.
|
||||
2. **State checkpointing & recovery** — thread JSONL (Amp), SQLite replication (celld), actor state persistence (rivet), VM disk snapshots (hypeman). Everything is replayable; nothing is lost on compute failure.
|
||||
|
||||
## Tool interaction: code as the tool interface
|
||||
|
||||
Three independent sources converge on the same insight:
|
||||
|
||||
- **Cloudflare Code Mode**: present MCP tools as a generated TypeScript API and let the LLM write code that calls it. LLMs are trained on billions of lines of real code but only contrived tool-call token sequences. Writing code also skips feeding every intermediate tool result back through the context window.
|
||||
- **nanocodex**: typed `Turn`/`TurnResult` values + event stream over WebSocket instead of stdout parsing. "stdout parsing for agent loops breaks constantly in edge cases; typed events give reliable state checkpoints and clean prompt cache hits."
|
||||
- **Prime Agent RLM**: the model's only tool is a persistent IPython REPL; context is a variable, subagents are function calls.
|
||||
|
||||
The unifying pattern: **the harness should look like a programming environment, not a tool-call menu.** This matters directly for how we build agent environments on hermes-vm (and why pi's minimalism / 4-tool philosophy won).
|
||||
|
||||
## What this means for hermes-vm (our stack)
|
||||
|
||||
Already have (the "durable compute host" layer):
|
||||
|
|
@ -52,7 +44,6 @@ Not worth building on a single box:
|
|||
|
||||
Interesting but optional:
|
||||
- **hypeman** — if we ever want real per-task VM isolation with snapshot/standby on our own box, hypeman is the lean OSS option (multi-hypervisor, OCI images, ms restore). Watch it; don't adopt yet.
|
||||
- **Cloudflare Code Mode pattern** — worth revisiting for our Effect TS stack: present our MCP/REST tools as a typed API and let agents write code against them, rather than exposing raw tool-call schemas.
|
||||
|
||||
## Verdict
|
||||
|
||||
|
|
|
|||
|
|
@ -1,41 +1,39 @@
|
|||
---
|
||||
title: Durable Agent Runtimes & Harness Engineering
|
||||
description: Survey of durable, stateful agent execution primitives — durable objects (celld), actors (rivet), multi-hypervisor VM runtimes (hypeman, E2B/Amp Orbs), harness-as-library (nanocodex), code-as-tool-calling (Cloudflare Code Mode) — and what they mean for building a self-hosted orb-like runtime on hermes-vm.
|
||||
title: Durable Agent Runtime
|
||||
description: State & durability lens of agent infrastructure — durable objects (celld), actors (rivet), multi-hypervisor VM runtimes (hypeman, E2B/Amp Orbs), scale-to-zero/hibernation — and what they mean for building a self-hosted orb-like runtime on hermes-vm. Sibling of harness-engineering under the agent-infra umbrella.
|
||||
status: active
|
||||
category: technical
|
||||
tags: [durable-objects, actors, agent-runtime, agent-harness, sandbox, microvm, self-hosting, orbs, stateful, durable-execution]
|
||||
tags: [durable-objects, actors, agent-runtime, agent-infra, sandbox, microvm, self-hosting, orbs, stateful, durable-execution]
|
||||
draft: true
|
||||
created: 2026-08-07
|
||||
updated: 2026-08-07
|
||||
updated: 2026-08-11
|
||||
origin: buzz://d8a718be-031f-4a6e-9f5c-a55466641654/4123d35532d0b2cbfb80a0a34f79fb3e4e63c16c25b77d43bc0f10bb178e4f1b
|
||||
---
|
||||
|
||||
# Durable Agent Runtimes & Harness Engineering
|
||||
# Durable Agent Runtime
|
||||
|
||||
## Summary
|
||||
|
||||
Research into how modern agent execution environments achieve **durability and statefulness** — where agent state lives, what compute primitive runs the agent, and how the environment hibernates and wakes. Covers the spectrum from in-process harness libraries (nanocodex) and V8 isolates (celld, Cloudflare Workers) to actor runtimes (rivet) and microVM sandboxes (hypeman, E2B/Amp Orbs). Goal: derive a build plan for a self-hosted, orb-like runtime on our NixOS VM (hermes-vm) without overbuilding.
|
||||
Research into how modern agent execution environments achieve **durability and statefulness** — where agent state lives, what compute primitive runs the agent, and how the environment hibernates and wakes. Covers durable objects / durable execution (celld, rivet, Durable Objects), actor runtimes, and microVM sandboxes (hypeman, E2B/Amp Orbs). The harness/loop lens (nanocodex, pi, Code Mode) now lives in [[harness-engineering/index|Harness Engineering]]. Goal: derive a build plan for a self-hosted, orb-like runtime on our NixOS VM (hermes-vm) without overbuilding.
|
||||
|
||||
## Research Question
|
||||
|
||||
What are the architectural primitives for building a durable, stateful agent runtime (sandboxed or not), and which combination fits a single self-hosted NixOS dev VM?
|
||||
What are the architectural primitives for a durable, stateful agent runtime (sandboxed or not), and which combination fits a single self-hosted NixOS dev VM?
|
||||
|
||||
## Scope
|
||||
|
||||
- Included: durable objects / durable execution (celld, rivet, Durable Objects), agent harness engineering (nanocodex, Amp orbs, Prime Agent RLM), sandbox runtimes (hypeman, E2B, Cloudflare isolates), tool-calling vs code-calling (Cloudflare Code Mode)
|
||||
- Excluded: model training/inference infra, prompt engineering, RLHF
|
||||
- Included: durable objects / durable execution (celld, rivet, Durable Objects), actor runtimes, sandbox/VM runtimes (hypeman, E2B, Cloudflare isolates), scale-to-zero/hibernation, orbs
|
||||
- Excluded (moved): agent harness/loop engineering → [[harness-engineering/index|Harness Engineering]]; model training/inference infra, prompt engineering, RLHF
|
||||
- Sufficient answer: a taxonomy of primitives + a recommendation for what to build (and what NOT to build) on hermes-vm
|
||||
|
||||
## Key Findings
|
||||
|
||||
- **The 2026 pattern is unanimous**: agent compute is disposable, agent state is durable elsewhere. Amp: sandbox is ephemeral, the thread (prompts, tool calls, results) lives on the server. celld: nodes are replaceable, the S3 bucket is the source of truth. Rivet: actor state persists while the process may hibernate.
|
||||
- **Four primitive families emerged**:
|
||||
- **Three primitive families emerged** (harness libraries split out to [[harness-engineering/index|Harness Engineering]]):
|
||||
1. Durable object/execution stores — celld (each object = its own SQLite DB, S3 replication, no consensus)
|
||||
2. Actor runtimes — rivet (in-memory state + persistence, ~20ms cold start, scale to zero, queues/workflows/scheduling)
|
||||
3. Sandbox/VM runtimes — hypeman (multi-hypervisor OCI-in-VM), E2B (Firecracker microVMs, snapshot restore, used as Amp's orb substrate)
|
||||
4. Harness libraries / agent loops — nanocodex (Rust, agent as in-process library, typed events over WebSocket), Codex, pi, Prime Agent RLM
|
||||
- **Scale-to-zero / hibernation is universal**: Amp orbs pause after 5 min idle; celld cells hibernate; rivet actors hibernate when idle; hypeman standby = snapshot VM to disk, resume in milliseconds; E2B snapshot reuse gives sub-30ms cold starts.
|
||||
- **Tool interaction is converging on "code as the tool interface"**: Cloudflare Code Mode converts MCP tools into a TypeScript API and has the LLM write code that calls it (LLMs are far better at code than at synthetic tool-call tokens). nanocodex's thesis is the same: typed events beat stdout parsing. Prime Agent's RLM: a persistent IPython REPL is the model's only tool. pi's minimalism: 4 tools, no MCP bloat.
|
||||
- **Two hard problems everyone solves**: (a) cold start latency (actor ~20ms vs VM ~30s, mitigated by snapshot/pooling), (b) state checkpointing and recovery (SQLite replication, thread JSONL, VM snapshots, actor state persistence).
|
||||
- **For our stack**: hermes-vm is already a durable compute host (systemd services, kanban DB, session DBs, herdr panes). The real gap is a per-task disposable environment contract + event wake, not a distributed state layer. celld/rivet solve problems we don't have on a single box; hypeman is interesting for per-task VMs but likely overkill.
|
||||
|
||||
|
|
@ -43,8 +41,6 @@ What are the architectural primitives for building a durable, stateful agent run
|
|||
|
||||
For the full reference list see [[durable-agent-runtime/sources|Sources]].
|
||||
|
||||
- [[durable-agent-runtime/sources|Sources]] — all links, one-line descriptions
|
||||
|
||||
## Detail Files
|
||||
|
||||
- [[durable-agent-runtime/sources|Sources]] — full reference list
|
||||
|
|
@ -54,20 +50,21 @@ For the full reference list see [[durable-agent-runtime/sources|Sources]].
|
|||
|
||||
## Related Research
|
||||
|
||||
- [[agent-infra/index|Agent Infrastructure]] — umbrella topic
|
||||
- [[harness-engineering/index|Harness Engineering]] — agent loop & tool interface lens (split out 2026-08-11)
|
||||
- [[codex-app-server-byo-auth/index|Codex App Server & BYO Auth]] — model access & embedding protocols
|
||||
- Amp Orbs research note (workspace): `RESEARCH/AMP_ORBS_REMOTE_DEV.md`
|
||||
- Prime Agent / Continual Harness (arXiv 2605.09998) — self-improving harness state
|
||||
- Forgejo AI reviewer harness (workspace skill reference) — where nanocodex was first evaluated
|
||||
|
||||
## Discussion
|
||||
|
||||
Triggered from the Amp Orbs thread (2026-08-07). User connected the dots: orbs, nanocodex, durable objects (celld), actors (rivet), multi-hypervisor VM runtimes (hypeman), and Cloudflare Code Mode all belong to one research area — "durable object, stateful machine, harness engineering either sandboxed or not." Asked for centralized structured research he can digest, with the option of parallel Luna delegation. Completed directly (all primary sources already extracted; delegation channel has known instability).
|
||||
Triggered from the Amp Orbs thread (2026-08-07). User connected the dots: orbs, nanocodex, durable objects (celld), actors (rivet), multi-hypervisor VM runtimes (hypeman), and Cloudflare Code Mode all belong to one research area — "durable object, stateful machine, harness engineering either sandboxed or not." Asked for centralized structured research he can digest. Completed directly (all primary sources already extracted; delegation channel has known instability). On 2026-08-11 the topic was narrowed to the state/durability lens and placed under the agent-infra umbrella; harness engineering split out.
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [ ] Decide Phase 1 scope: per-repo `.agents/setup`-style contracts + AGENTS.md per repo on hermes-vm
|
||||
- [ ] Evaluate hypeman as an opt-in per-task VM runtime (snapshot/standby) vs NixOS containers
|
||||
- [ ] Revisit Code Mode pattern for our Effect TS stack (tools as typed API, not tool-calling)
|
||||
|
||||
## Conclusion
|
||||
|
||||
The research area is **durable, stateful agent runtime and harness engineering — sandboxed or not**. Four primitive families cover the space; for a single self-hosted NixOS VM the right move is environment contracts + event wake on the existing durable host, not building a distributed durable-object layer.
|
||||
The research area is **durable, stateful agent runtime — sandboxed or not**. Three primitive families cover the space; for a single self-hosted NixOS VM the right move is environment contracts + event wake on the existing durable host, not building a distributed durable-object layer. The tool-interface half of the original question now lives in [[harness-engineering/index|Harness Engineering]].
|
||||
|
|
|
|||
|
|
@ -1,18 +1,18 @@
|
|||
# Next Steps — Durable Agent Runtimes & Harness Engineering
|
||||
# Next Steps — Durable Agent Runtime
|
||||
|
||||
> Part of [[durable-agent-runtime/index|Durable Agent Runtimes & Harness Engineering]]
|
||||
> Part of [[durable-agent-runtime/index|Durable Agent Runtime]]
|
||||
|
||||
## Open questions
|
||||
|
||||
- [ ] Which repos get the `.agents/setup`-style contract first (clan-private, law retrieval, idx-cli)?
|
||||
- [ ] Should the dev-server + ports-file pattern be a shared script library in clan-private or per-repo?
|
||||
- [ ] Is hypeman worth an evaluation build on hermes-vm (per-task VM with standby) vs NixOS containers?
|
||||
- [ ] Revisit Cloudflare Code Mode for our Effect TS stack: tools-as-typed-API vs tool-calling.
|
||||
- [ ] Does the Forgejo webhook → kanban wake need a new service, or does an existing cron/webhook path cover it?
|
||||
- [ ] Code Mode pattern for our Effect TS stack now tracked in [[harness-engineering/index|Harness Engineering]]
|
||||
|
||||
## Decisions made (2026-08-07)
|
||||
|
||||
- Research topic: **durable-agent-runtime** — covers durable objects, stateful machines, and harness engineering (sandboxed or not).
|
||||
- Research topic: **durable-agent-runtime** — covers durable objects, stateful machines, and harness engineering (sandboxed or not). On 2026-08-11 harness engineering was split out into its own topic under the agent-infra umbrella.
|
||||
- Do NOT build a distributed durable-object layer (celld) or actor platform (rivet) on a single box.
|
||||
- Do NOT build an E2B-style per-thread VM fleet; use NixOS-native isolation when needed.
|
||||
- DO build environment contracts: per-repo setup hooks, AGENTS.md per repo, dev-server skills, event wake.
|
||||
|
|
|
|||
42
harness-engineering/analysis.md
Normal file
42
harness-engineering/analysis.md
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
# Analysis — Harness Engineering
|
||||
|
||||
> Part of [[harness-engineering/index|Harness Engineering]]
|
||||
|
||||
## Code as the tool interface
|
||||
|
||||
Three independent sources converge on the same insight:
|
||||
|
||||
- **Cloudflare Code Mode**: present MCP tools as a generated TypeScript API and let the LLM write code that calls it. LLMs are trained on billions of lines of real code but only contrived tool-call token sequences. Writing code also skips feeding every intermediate tool result back through the context window.
|
||||
- **nanocodex**: typed `Turn`/`TurnResult` values + event stream over WebSocket instead of stdout parsing. "stdout parsing for agent loops breaks constantly in edge cases; typed events give reliable state checkpoints and clean prompt cache hits."
|
||||
- **Prime Agent RLM**: the model's only tool is a persistent IPython REPL; context is a variable, subagents are function calls.
|
||||
|
||||
The unifying pattern: **the harness should look like a programming environment, not a tool-call menu.** This matters directly for how we build agent environments on hermes-vm (and why pi's minimalism / 4-tool philosophy won).
|
||||
|
||||
## Cloudflare Code Mode mechanics (2026-08-11)
|
||||
|
||||
Code Mode = let the model write code instead of calling tools one-by-one. The model gets ONE tool (code execution); its code becomes a compact plan that calls the real tools, processes results, and returns only what the model needs.
|
||||
|
||||
**Why it beats direct tool calling**
|
||||
- Direct tool calls: model picks tool → result goes back into context → model picks next tool → round-trip each time. Intermediate data pollutes context even when the model only needs a fragment.
|
||||
- Code Mode: all intermediate logic (compose calls, loop over results, filter/transform, branch) runs inside ONE sandboxed execution. Only the final value comes back to the model. Less context, fewer round-trips, more reusable.
|
||||
|
||||
**Key features**
|
||||
- **Tools as typed methods** — configured tools become a TypeScript API the model codes against
|
||||
- **Progressive discovery** — `codemode.search()` / `codemode.describe()` pull connector definitions on demand instead of loading a huge catalog into context every request (solves the "21-tool MCP eats 13.7k tokens" problem)
|
||||
- **Execution safety** — generated code runs in an isolated V8 sandbox (not containers); the sandbox's only exit is the typed tool API, so it can't leak keys or touch the network except through approved connectors
|
||||
- **Integration surfaces** — durable runtime (connectors + approvals + replay + snippets), AI SDK, TanStack AI, browser (isolated iframe), MCP (expose MCP tools inside Code Mode), OpenAPI (derive typed methods from an OpenAPI spec)
|
||||
|
||||
**When to use (their guidance)**
|
||||
- Direct tool calls: simple tasks, small fixed tool set
|
||||
- Code Mode: composed/dependent calls, large tool catalogs, loops/branching/filtering, reusable logic — i.e. multi-step agent work
|
||||
|
||||
**Caveats**
|
||||
- Marked **experimental** — fine to prototype with, caution for production
|
||||
|
||||
## Implications for our Effect TS stack
|
||||
|
||||
The MCP + OpenAPI surfaces are directly relevant: our Effect TS services could expose a typed API that agents *write code against* instead of raw tool-call schemas. Same convergence as the Codex app-server thread/turn/item streaming model and its typed client schemas (`codex app-server generate-ts`). Watch item: "Code Mode pattern for Effect TS".
|
||||
|
||||
## Verdict
|
||||
|
||||
Harness engineering is where most of the leverage lives for a single-box agent platform: typed events, streaming items, and a code-shaped tool interface cost little to adopt and remove the classic agent-loop failure modes (stdout parsing, context pollution, tool-catalog bloat).
|
||||
46
harness-engineering/index.md
Normal file
46
harness-engineering/index.md
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
---
|
||||
title: Harness Engineering — Agent Loops & Code-as-Tool-Interface
|
||||
description: The agent harness layer — how the agent loop is built and how the model interacts with tools. Covers nanocodex (agent as in-process library), pi minimalism, Codex protocols, and code-as-tool-interface (Cloudflare Code Mode: tools as typed API, progressive discovery, V8 sandbox).
|
||||
status: active
|
||||
category: technical
|
||||
tags: [harness, agent-loop, code-as-tool, codemode, cloudflare, nanocodex, pi, codex, tool-calling, typed-events, agent-infra]
|
||||
draft: true
|
||||
created: 2026-08-11
|
||||
updated: 2026-08-11
|
||||
origin: buzz://d8a718be-031f-4a6e-9f5c-a55466641654/c66f017974f748bd6db019c913d3a6f59de349e0cbd90ae38c222e9640521d63
|
||||
---
|
||||
|
||||
# Harness Engineering — Agent Loops & Code-as-Tool-Interface
|
||||
|
||||
## Summary
|
||||
|
||||
The **harness** is the layer that runs the agent loop: it connects the model to tools, manages context, and streams events. Split out of durable-agent-runtime (2026-08-11) because it is a distinct lens — durability is about state; harness engineering is about the loop and the tool interface. Central 2026 finding: **code-as-tool-interface** — the harness should look like a programming environment, not a tool-call menu.
|
||||
|
||||
## Key Findings
|
||||
|
||||
- **Code as the tool interface is the converging pattern** (three independent sources):
|
||||
- Cloudflare Code Mode: MCP tools become a generated TypeScript API; the model writes code that calls it
|
||||
- nanocodex: typed `Turn`/`TurnResult` events over WebSocket beat stdout parsing
|
||||
- Prime Agent RLM: a persistent IPython REPL is the model's only tool
|
||||
- **Why code beats direct tool calls**: LLMs are trained on real code, not synthetic tool-call tokens; intermediate results don't pollute context; composed logic is reusable.
|
||||
- **Cloudflare Code Mode mechanics** (see analysis): tools as typed methods, progressive discovery (`codemode.search()` / `codemode.describe()`), execution safety via V8 isolate, six integration surfaces, experimental status.
|
||||
- **Minimalism wins for small harnesses**: pi's 4-tool philosophy; don't load a huge MCP catalog into context.
|
||||
|
||||
## Sources
|
||||
|
||||
Primary: https://developers.cloudflare.com/agents/tools/codemode/ (2026-08-11); https://learn.chatgpt.com/docs/app-server; nanocodex repo; Prime Agent RLM (arXiv 2605.09998). See [[durable-agent-runtime/sources|Durable Agent Runtime sources]] for shared references.
|
||||
|
||||
## Detail Files
|
||||
|
||||
- [[harness-engineering/analysis|Analysis]] — code-as-tool-interface deep-dive, Cloudflare Code Mode mechanics, implications for our Effect TS stack
|
||||
|
||||
## Related Research
|
||||
|
||||
- [[agent-infra/index|Agent Infrastructure]] — umbrella topic
|
||||
- [[durable-agent-runtime/index|Durable Agent Runtime]] — state/durability lens (formerly contained this topic)
|
||||
- [[codex-app-server-byo-auth/index|Codex App Server & BYO Auth]] — Codex app-server as an embedding protocol surface
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [ ] Prototype the "Code Mode pattern for Effect TS": expose our MCP/REST tools as a typed API agents write code against
|
||||
- [ ] Evaluate which harness primitives (typed events, streaming items, code sandbox) belong in our own agent tooling
|
||||
Loading…
Add table
Add a link
Reference in a new issue