0xrsydn/idx-cli
1
mirror of https://github.com/0xrsydn/idx-cli.git synced 2026-11-07 09:45:36 +00:00
Code Issues Projects Releases 1 Packages Wiki Activity

test(agents): close ways an --npx run could pass or fail wrongly

Browse source 
From a Codex review of the harness:
- Drop target/debug from PATH in --npx mode (the dev shell adds it), so
  a missing package bin cannot fall through to the local build, and fail
  an agent whose npm cache holds no installed idx-mcp binary.
- The live `changes` case must diff the two latest releases, read from
  the synced snapshot, not any two dates.
- Hermes: `hermes -z` waits ~15s for MCP tools regardless of
  connect_timeout, so a slow cold install lost the first case. Warm up
  with `hermes mcp test idx`, which installs through Hermes' own npx and
  honors connect_timeout (now 300s).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
This commit is contained in:
hermes 2026-10-16 08:03:39 +00:00
commit 687c9ad83f
2 changed files with 33 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

2
tests/agents/cases.json
View file

@ -36,7 +36,7 @@
"expect_text": [],
"live": {
"prompt": "Which shareholders changed between the two most recent KSEI releases?",
"expect_tools": [{"name": "get_ownership_changes"}]
"expect_tools": [{"name": "get_ownership_changes", "args": {"from": "{previous_release}", "to": "{latest_release}"}}]
}
}
]