mirror of
https://github.com/0xrsydn/idx-cli.git
synced 2026-11-07 09:45:36 +00:00
test: add golden output parity gate and live A/B compare
Phase 0 of docs/DESIGN.md, before any refactor code: - tests/golden.rs runs every case in tests/golden/cases.toml (126 outputs across help, completions, config, cache, all stocks commands on MSN and Yahoo mocks, error paths, and ownership queries on a seeded fixture DB) and compares exit code, stdout, and stderr byte for byte. Regenerate only with UPDATE_GOLDEN=1 for a reviewed output change. - scripts/live-smoke.sh --compare <old-idx> runs each live case with both binaries and fails on differing exit codes, added/missing/retyped JSON keys, or changed values for slow-moving data. - Exclude golden files from the whitespace/EOF pre-commit fixers, which would otherwise rewrite pinned help output. Both gates were fault-injected: a one-word table change fails exactly the three history goldens, and a binary dropping profile.long_name fails only the profile live comparison. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
29796f0f9d
commit
fe627496dd
133 changed files with 6033 additions and 23 deletions
|
|
@ -13,7 +13,7 @@ src/
|
|||
├── ownership/ xlsx + pdf parsers, archive (ZIP/TXT), remote discovery,
|
||||
│ snapshot sync, db (SQLite/FTS5), entities, search, graph
|
||||
└── output/ table.rs, json.rs
|
||||
tests/cli.rs CLI integration tests; fixtures in tests/fixtures/
|
||||
tests/ cli.rs (behavior), golden.rs + golden/ (output parity), fixtures/
|
||||
scripts/ smoke, installer, and ownership snapshot publisher
|
||||
```
|
||||
|
||||
|
|
@ -42,5 +42,5 @@ Docs:
|
|||
```bash
|
||||
nix flake check -L # canonical gate (fmt, clippy, tests, package)
|
||||
nix develop --command cargo test <name> # focused loop
|
||||
scripts/live-smoke.sh # opt-in live provider drift check
|
||||
scripts/live-smoke.sh --compare <old> # live drift check, A/B against an older build
|
||||
```
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue