- Require bash 4.4+ up front, and match answer facts with jq regexes
(Oniguruma, (?i) everywhere) instead of GNU-only grep -P.
- Unparseable agent output fails that run instead of aborting the
matrix; check_run now also reports unparseable tool calls, which
previously passed the tool check silently.
- Copy the Hermes model block with awk up to the next top-level key, so
a final or differently ordered block is handled.
- --keep keeps run output but deletes the copied agent credentials.
- docs: full path to the tool golden.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- tests/agents/agent-smoke.sh runs tests/agents/cases.json through
Claude Code, Codex, pi, Hermes Agent, and opencode headless against a
local idx-mcp build, each with an isolated config, and checks the
tools called, their key arguments, and (mock data) facts in the
answer. Opt-in; never part of the gate.
- crates/idx-mcp/tests/golden/tools.json pins tool names, descriptions,
and input schemas, so changes to what agents see are reviewed.
- nix develop .#agents adds Claude Code, Codex, and opencode from
nixpkgs after the host PATH.
First run: 30/30 (5 agents x 6 cases, mock data).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Phase 3a of docs/DESIGN.md. MockProvider moves to api/mock.rs, and it,
the embedded MSN fixture bodies, and the IDX_MOCK_* hooks compile only
with idx-core's new `testing` feature. idx-cli enables it as a
dev-dependency, so `cargo test` keeps the fixture-backed provider while
release binaries no longer embed fixture data or honor
IDX_USE_MOCK_PROVIDER. Parser helpers used only by the mock are gated
the same way.
The packaged-cli flake check no longer relies on the mock; it checks
an offline cache miss (JSON CACHEMISS, exit 3) instead.
Goldens unchanged; clippy is clean with and without `testing`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Phase 0 of docs/DESIGN.md, before any refactor code:
- tests/golden.rs runs every case in tests/golden/cases.toml (126
outputs across help, completions, config, cache, all stocks commands
on MSN and Yahoo mocks, error paths, and ownership queries on a seeded
fixture DB) and compares exit code, stdout, and stderr byte for byte.
Regenerate only with UPDATE_GOLDEN=1 for a reviewed output change.
- scripts/live-smoke.sh --compare <old-idx> runs each live case with
both binaries and fails on differing exit codes, added/missing/retyped
JSON keys, or changed values for slow-moving data.
- Exclude golden files from the whitespace/EOF pre-commit fixers, which
would otherwise rewrite pinned help output.
Both gates were fault-injected: a one-word table change fails exactly
the three history goldens, and a binary dropping profile.long_name fails
only the profile live comparison.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 43-case mock smoke duplicated tests/cli.rs, which already runs the
real binary against fixture providers. Replace the flake's smoke-mock
check with a small packaged-cli check (version JSON, bash completions,
one mock quote), and cut scripts/live-smoke.sh from 672 to ~170 lines:
it now only runs shipped commands against live MSN/Yahoo/IDX and
validates output shape. Rewrite docs/SMOKE.md around the Nix gate and
the drift check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Remove FEATURE_SPEC.md, TODO.md, and docs/AGENT_INTEGRATION.md; their
core gaps have shipped. Open work moves to a short ROADMAP.md.
- Remove the CLAUDE.md symlink and rewrite AGENTS.md as a quick guide:
navigation, rules, and the nix flake check gate.
- Merge OWNERSHIP_SELF_HOSTED.md into OWNERSHIP_PUBLISH.md and drop
private host integration notes.
- ARCHITECTURE: add MSN endpoints, clarify Bing ingest is unsupported.
- OWNERSHIP_SYNC: note source.pdf_url also carries XLSX URLs.
- systemd example: --history 2 so snapshots stay under the 10 MiB
client cap.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Describe canonical local and CI verification, cache reuse, output assertions, workflow boundaries, and observed regression results without treating test counts as coverage targets.
Clear the smoke cache before each cache-group warm case so the following offline and stale-cache checks always start from a fresh baseline instead of earlier group state.
Document the runner behavior in the smoke guide to make the cache-group sequencing explicit.
