The `changes` case names the fixture releases (2026-01-30, 2026-02-27),
which the live snapshot does not have, so under --live/--npx it failed
agents that correctly declined to answer about missing releases and
passed ones that called the tool anyway. A case's `live` fields now
replace the mock ones; this one asks about the two most recent releases.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
- tests/agents/agent-smoke.sh runs tests/agents/cases.json through
Claude Code, Codex, pi, Hermes Agent, and opencode headless against a
local idx-mcp build, each with an isolated config, and checks the
tools called, their key arguments, and (mock data) facts in the
answer. Opt-in; never part of the gate.
- crates/idx-mcp/tests/golden/tools.json pins tool names, descriptions,
and input schemas, so changes to what agents see are reviewed.
- nix develop .#agents adds Claude Code, Codex, and opencode from
nixpkgs after the host PATH.
First run: 30/30 (5 agents x 6 cases, mock data).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
