# Smoke Checks Use `scripts/live-smoke.sh` as the reusable smoke runner for shipped CLI surfaces. The script: - builds `target/debug/idx` once by default - isolates config, cache, and data under `tmp/live-smoke//` - records separate `.stdout` and `.stderr` files per case under `tmp/live-smoke//logs/`, plus `.validation` diagnostics for JSON checks - supports fast mock-backed runs and slower live-network passes; `jq` is required and provided by `nix develop` ## Common Runs ```bash scripts/live-smoke.sh scripts/live-smoke.sh --mode full scripts/live-smoke.sh --mode mock scripts/live-smoke.sh --group live-table --group live-json scripts/live-smoke.sh --group live-nonfinite scripts/live-smoke.sh --group cache --symbol BBRI scripts/live-smoke.sh --dry-run --mode full scripts/live-smoke.sh --bin ./tmp/release-install/bin/idx --no-build --mode mock scripts/audit-msn-fundamentals.sh --tickers BUMI,ADRO,AIMS ``` ## Nix CI Checks `nix flake check -L` is the canonical local, pre-push, and Linux CI gate. Its eleven checks cover: - Rust formatting, Clippy (`--all-targets`), and all unit/CLI tests - the optimized application build and verified crate packaging - the packaged ownership publisher's `--help` entry point - the offline mock CLI smoke matrix - publisher regression scenarios, including the SQLite artifact builder - installer regressions under `sh` and `dash`, plus installer and smoke-runner ShellCheck All check tools are pinned by `flake.lock`. Installer fixtures use resolved shell paths, and generated publisher stubs use the running Bash interpreter, so the regression suites also work inside the Nix sandbox. Crane shares dev-profile dependency artifacts between tests, Clippy, and `cargo package` verification; the shipped application uses a separate release dependency cache. Package verification still builds the packaged crate. Integration-test edits do not invalidate the application build, and script checks consume only their relevant source files. README/license edits affect the packaging check, not the Rust test/application derivations. CI cache keys include platform, dependency/toolchain inputs, and checked source contents. Source changes can therefore save new outputs; restore prefixes first reuse the same dependency/toolchain cache, then another cache for the platform. The GitHub cache is opportunistic and subject to eviction, not a shared public binary cache. Obsolete PR runs are cancelled; main-branch runs are not. Use individual checks when investigating a failure (replace the system as needed), or targeted Cargo commands inside the pinned development shell: ```bash nix build .#checks.x86_64-linux.publisher-test -L nix develop --command cargo test full_screener ``` The native macOS installer job remains separate to exercise BSD/macOS tools. Live-network smoke is deliberately outside the sandboxed offline gate. For a live full-list screener check, use an isolated config/cache and run: ```bash idx -o json stocks screen --region us --filter top-performers --limit 600 idx --offline -o json stocks screen --region us --filter top-performers --limit 3 idx --offline -o json stocks screen --region us --filter top-performers --limit 25 ``` The first command should not stop at the old 500-row cap when MSN reports more candidates. The two offline results must match prefixes of the same cached full list. Provider counts and prices vary; live HTTP failures are not offline-test failures. ## Coverage Boundaries The mock preset has 43 cases: 7 general, 19 stock, 9 cache, 2 routing, 4 error, and 2 empty/unsupported ownership checks. This is a packaged-binary safety net, not a claim that every important behavior is covered by shell smoke. - All fifteen stock commands run in JSON mode with structural and fixture-value assertions. Four representative table renders check displayed data. - JSON checks require exactly one document in the correct stream. Failed commands must leave stdout empty and expose the expected stable error code. - Cache scenarios use zero TTL, force provider failures, and compare parsed offline/stale results with the warmed data; diagnostics must stay on stderr. - Redundant table launches, duplicate routing cases, and repeated setup checks are omitted rather than counted as extra correctness coverage. The CLI integration suite carries deeper workflows under the same Nix gate: cache misses, provider isolation, filtered reports, non-finite values, XLSX/ZIP import and ownership queries, snapshot install/no-op/force, and rejected updates preserving database bytes and query results. Screener regressions generate 503 distinct candidates, with ranking leaders beyond the first 500. They exercise top/worst performers, volume, and market-cap ordering, a small first request followed by complete offline reads, and rejected missing/incomplete totals without cache poisoning. The raw fixture override `IDX_MOCK_MSN_SCREENER_FIXTURE` is honored only with `IDX_USE_MOCK_PROVIDER`; responses honor the requested batch size so the complete-fetch path is exercised. Mocks do not verify current upstream availability, authentication, or response drift. Live provider checks remain a separate, opt-in operation. ## Modes - `live` runs the default real-network baseline: `general`, `live-table`, and `ownership` - `mock` runs the deterministic baseline: `general`, `mock`, `cache`, `routing`, `errors`, and `ownership` - `full` runs every group, including live JSON output checks and ownership-import verification ## Groups - `general`: `version`, `completions`, `config`, and `cache` command basics - `live-table`: all shipped `stocks` commands in live table mode - `live-json`: all shipped `stocks` commands in live JSON mode - `mock`: all shipped `stocks` commands against the mock provider in JSON mode, plus representative quote/history/fundamental/financials table renders - `cache`: exact data equality across warm, `--offline`, and stale-cache fallback reads for quote, technical, and MSN `profile` - `routing`: Yahoo/MSN provider routing plus explicit MSN history behavior - `errors`: JSON error contract and invalid flag/input checks - `live-nonfinite`: opt-in live MSN fundamentals checks for known non-finite ticker payloads (`BUMI`, `ADRO`, `AIMS`) - `ownership`: safe ownership smoke checks that do not require imported ownership data - `ownership-import`: live discovery/import hardening checks for supported `above1` import plus expected unsupported legacy-family failures ## Notes - The runner forces `IDX_OUTPUT=table` as its default environment so table cases stay stable; JSON checks use `-o json` explicitly. - Cache-group warm cases clear the smoke cache before they run so each warm/offline/stale sequence starts clean and stale-cache assertions are not masked by earlier groups. - Use `--bin --no-build` when you want to validate an installed binary instead of the workspace `target/debug/idx` build. - Use `scripts/audit-msn-fundamentals.sh` for a full CLI valuation sweep across the IDX MSN symbol map. It is intentionally separate from the reusable smoke runner because it is a heavier provider-health audit, not a stable baseline check. - Ownership commands that need imported data are intentionally not part of the baseline runner yet. The current baseline only covers `ownership releases` and the known unsupported `ownership import --fetch-bing`. - `ownership sync` is still primarily covered by fixture-backed CLI tests rather than the reusable smoke runner. - The new `ownership-import` group is intentionally opt-in for explicit `--group ownership-import` runs or `--mode full`; it discovers live URLs first, imports the supported `above1` attachment into the temp DB, then asserts the current `above5` and `investor-type` URLs fail with explicit unsupported-schema UX. - The new `live-nonfinite` group is intentionally opt-in only. As of `2026-04-13`, the known real repro tickers are `BUMI`, `ADRO`, and `AIMS`. - When a case fails, inspect its `.stdout`, `.stderr`, and (for JSON cases) `.validation` files in `tmp/live-smoke/.../logs/` before updating `TODO.md` or `FEATURE_SPEC.md`.