mirror of
https://github.com/0xrsydn/idx-cli.git
synced 2026-11-07 09:45:36 +00:00
e78d9fcecd
Phase 3a of docs/DESIGN.md. MockProvider moves to api/mock.rs, and it, the embedded MSN fixture bodies, and the IDX_MOCK_* hooks compile only with idx-core's new `testing` feature. idx-cli enables it as a dev-dependency, so `cargo test` keeps the fixture-backed provider while release binaries no longer embed fixture data or honor IDX_USE_MOCK_PROVIDER. Parser helpers used only by the mock are gated the same way. The packaged-cli flake check no longer relies on the mock; it checks an offline cache miss (JSON CACHEMISS, exit 3) instead. Goldens unchanged; clippy is clean with and without `testing`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
68 lines
3.3 KiB
Markdown
68 lines
3.3 KiB
Markdown
# Testing and Smoke Checks
|
|
|
|
## Automated gate
|
|
|
|
`nix flake check -L` is the canonical local, pre-push, and CI gate. It runs
|
|
offline with pinned tools:
|
|
|
|
| Check | Covers |
|
|
| --- | --- |
|
|
| `fmt`, `clippy` | formatting and lints (`--all-targets`) |
|
|
| `test` | unit/module tests, `tests/cli.rs` CLI integration tests, and `tests/golden.rs` output parity |
|
|
| `default`, `package` | release build and `cargo package` verification |
|
|
| `packaged-cli` | the wrapped Nix release binary emits JSON, generates completions, and returns a JSON `CACHEMISS` error with exit 3 offline |
|
|
| `ownership-publisher`, `publisher-test` | packaged publisher entry point and its regression scenarios |
|
|
| `install-script-sh`, `install-script-dash`, `shellcheck` | installer regressions and shell lint |
|
|
|
|
`tests/cli.rs` is the behavioral suite: it runs the real `idx` binary against
|
|
fixture-backed mock providers (`IDX_USE_MOCK_PROVIDER=1`) and covers output
|
|
contracts, cache/offline transitions, provider routing, error codes, ownership
|
|
imports and queries, and snapshot sync.
|
|
|
|
`tests/golden.rs` pins the exact output (exit code, stdout, stderr) of every
|
|
command listed in `tests/golden/cases.toml`, in table and JSON. A mismatch means
|
|
user-visible output changed. Regenerate with
|
|
`UPDATE_GOLDEN=1 cargo test --test golden` only when the change is intended,
|
|
and commit the golden diff with the reason.
|
|
|
|
For a focused loop, run one check or filter Cargo tests:
|
|
|
|
```bash
|
|
nix build .#checks.x86_64-linux.publisher-test -L
|
|
nix develop --command cargo test full_screener
|
|
```
|
|
|
|
CI cache keys include platform, dependency/toolchain inputs, and checked source
|
|
contents. The native macOS installer job runs separately to exercise BSD tools.
|
|
|
|
## Live drift check
|
|
|
|
Fixtures cannot tell when MSN, Yahoo, or IDX change their responses.
|
|
`scripts/live-smoke.sh` runs shipped commands against the real upstreams and
|
|
validates each JSON output shape. It is opt-in and never part of the gate.
|
|
|
|
```bash
|
|
nix develop --command scripts/live-smoke.sh # stocks + ownership
|
|
nix develop --command scripts/live-smoke.sh --group stocks --symbol BBRI
|
|
nix develop --command scripts/live-smoke.sh --bin ./result/bin/idx
|
|
nix develop --command scripts/live-smoke.sh --compare ./old/bin/idx # A/B against an older build
|
|
scripts/live-smoke.sh --dry-run
|
|
```
|
|
|
|
- `stocks`: every `stocks` command for one ticker, plus `compare BUMI,ADRO,AIMS`,
|
|
whose MSN key ratios contain non-finite values.
|
|
- `ownership`: `discover`, then imports the newest supported report into a temp
|
|
DB and lists `releases`. IDX sits behind Cloudflare and returns `403` to many
|
|
datacenter IPs, including GitHub-hosted runners; run it from a network IDX accepts.
|
|
|
|
`--compare <old-idx>` runs every case with both binaries back to back, each
|
|
with its own config and cache, and fails on a different exit code, any added,
|
|
missing, or retyped JSON key, or changed values for `profile`, `financials`,
|
|
`earnings`, and ownership results. Build the old side with, for example,
|
|
`nix build "git+file://$PWD?ref=main" -o old`.
|
|
|
|
Each run isolates config, cache, and data under `tmp/live-smoke/<timestamp>/`
|
|
and keeps per-case `.stdout`, `.stderr`, and `.validation` logs there.
|
|
|
|
`scripts/audit-msn-fundamentals.sh` is a heavier sweep of valuation output
|
|
across the whole IDX MSN symbol map, for provider-health audits.
|