Commit graph

9 commits

Author SHA1 Message Date
hermes
fe627496dd test: add golden output parity gate and live A/B compare
Phase 0 of docs/DESIGN.md, before any refactor code:

- tests/golden.rs runs every case in tests/golden/cases.toml (126
  outputs across help, completions, config, cache, all stocks commands
  on MSN and Yahoo mocks, error paths, and ownership queries on a seeded
  fixture DB) and compares exit code, stdout, and stderr byte for byte.
  Regenerate only with UPDATE_GOLDEN=1 for a reviewed output change.
- scripts/live-smoke.sh --compare <old-idx> runs each live case with
  both binaries and fails on differing exit codes, added/missing/retyped
  JSON keys, or changed values for slow-moving data.
- Exclude golden files from the whitespace/EOF pre-commit fixers, which
  would otherwise rewrite pinned help output.

Both gates were fault-injected: a one-word table change fails exactly
the three history goldens, and a binary dropping profile.long_name fails
only the profile live comparison.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 08:33:25 +00:00
hermes
86c108f6eb test: replace mock smoke matrix with a live-only drift check
The 43-case mock smoke duplicated tests/cli.rs, which already runs the
real binary against fixture providers. Replace the flake's smoke-mock
check with a small packaged-cli check (version JSON, bash completions,
one mock quote), and cut scripts/live-smoke.sh from 672 to ~170 lines:
it now only runs shipped commands against live MSN/Yahoo/IDX and
validates output shape. Rewrite docs/SMOKE.md around the Nix gate and
the drift check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 07:45:51 +00:00
hermes
26ffd445e8 ci(nix): unify cached verification and strengthen packaged smoke
Run deterministic installer and publisher checks in Nix, align dev and release dependency artifacts, narrow derivation inputs, and use content-aware CI cache keys. Replace duplicate Cargo gates with a Nix pre-push gate; validate JSON streams and cache data in representative packaged-binary smoke cases.
2026-10-03 16:22:11 +00:00
a2d15d2125 feat: support explicit MSN chart history 2026-04-15 13:48:27 +07:00
19748d397e fix: handle non-finite msn key ratios 2026-04-13 23:34:36 +07:00
c94c826307 chore: prepare v0.2.1 release 2026-04-06 17:47:20 +07:00
a52712b063 Complete batch 2 ownership import hardening 2026-03-30 16:40:32 +07:00
7928e83cc5 test: reset smoke cache between warm cases
Clear the smoke cache before each cache-group warm case so the following offline and stale-cache checks always start from a fresh baseline instead of earlier group state.

Document the runner behavior in the smoke guide to make the cache-group sequencing explicit.
2026-03-30 12:35:22 +07:00
d23f16e7a3 test: live smoke testing 2026-03-27 16:53:57 +07:00