0xrsydn/idx-cli
1
mirror of https://github.com/0xrsydn/idx-cli.git synced 2026-11-07 01:35:36 +00:00
Code Issues Projects Releases 1 Packages Wiki Activity
idx-cli/docs/SMOKE.md
hermes 86c108f6eb test: replace mock smoke matrix with a live-only drift check 
The 43-case mock smoke duplicated tests/cli.rs, which already runs the
real binary against fixture providers. Replace the flake's smoke-mock
check with a small packaged-cli check (version JSON, bash completions,
one mock quote), and cut scripts/live-smoke.sh from 672 to ~170 lines:
it now only runs shipped commands against live MSN/Yahoo/IDX and
validates output shape. Rewrite docs/SMOKE.md around the Nix gate and
the drift check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 07:45:51 +00:00

2.4 KiB
Raw Blame History

Testing and Smoke Checks

Automated gate

nix flake check -L is the canonical local, pre-push, and CI gate. It runs offline with pinned tools:

Check Covers
fmt, clippy formatting and lints (--all-targets)
test unit/module tests and tests/cli.rs CLI integration tests
default, package release build and cargo package verification
packaged-cli the wrapped Nix binary starts, emits JSON, and generates completions
ownership-publisher, publisher-test packaged publisher entry point and its regression scenarios
install-script-sh, install-script-dash, shellcheck installer regressions and shell lint

tests/cli.rs is the behavioral suite: it runs the real idx binary against fixture-backed mock providers (IDX_USE_MOCK_PROVIDER=1) and covers output contracts, cache/offline transitions, provider routing, error codes, ownership imports and queries, and snapshot sync.

For a focused loop, run one check or filter Cargo tests:

nix build .#checks.x86_64-linux.publisher-test -L
nix develop --command cargo test full_screener

CI cache keys include platform, dependency/toolchain inputs, and checked source contents. The native macOS installer job runs separately to exercise BSD tools.

Live drift check

Fixtures cannot tell when MSN, Yahoo, or IDX change their responses. scripts/live-smoke.sh runs shipped commands against the real upstreams and validates each JSON output shape. It is opt-in and never part of the gate.

nix develop --command scripts/live-smoke.sh                   # stocks + ownership
nix develop --command scripts/live-smoke.sh --group stocks --symbol BBRI
nix develop --command scripts/live-smoke.sh --bin ./result/bin/idx
scripts/live-smoke.sh --dry-run
  • stocks: every stocks command for one ticker, plus compare BUMI,ADRO,AIMS, whose MSN key ratios contain non-finite values.
  • ownership: discover, then imports the newest supported report into a temp DB and lists releases. IDX sits behind Cloudflare and returns 403 to many datacenter IPs, including GitHub-hosted runners; run it from a network IDX accepts.

Each run isolates config, cache, and data under tmp/live-smoke/<timestamp>/ and keeps per-case .stdout, .stderr, and .validation logs there.

scripts/audit-msn-fundamentals.sh is a heavier sweep of valuation output across the whole IDX MSN symbol map, for provider-health audits.