test(agents): run agents from an empty directory, not the checkout

From the checkout, agents read its AGENTS.md and docs: Hermes concluded
from docs/IDX_NEWS.md that news "is not implemented" and failed the news
case it had passed before the doc existed. Users' agents never see our
repo, so run every agent from an empty directory under the work dir.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
This commit is contained in:
hermes 2026-10-16 09:53:31 +00:00
commit 6f44c3ce50

View file

@ -360,6 +360,12 @@ if [[ ${#only_cases[@]} -gt 0 ]]; then
case_ids=("${only_cases[@]}") case_ids=("${only_cases[@]}")
fi fi
# Run agents from an empty directory: from the checkout, agents read its
# AGENTS.md and docs (Hermes concluded from docs/IDX_NEWS.md that news "is not
# implemented"), which users' agents never see.
mkdir -p "$work/cwd"
cd "$work/cwd"
results="$work/results.jsonl" results="$work/results.jsonl"
: > "$results" : > "$results"
echo "agent-smoke: agents=${agents[*]} cases=${#case_ids[@]} runs=$runs" echo "agent-smoke: agents=${agents[*]} cases=${#case_ids[@]} runs=$runs"