0xrsydn/idx-cli
1
mirror of https://github.com/0xrsydn/idx-cli.git synced 2026-11-07 01:35:36 +00:00
Code Issues Projects Releases 1 Packages Wiki Activity

test: replace mock smoke matrix with a live-only drift check

Browse source 
The 43-case mock smoke duplicated tests/cli.rs, which already runs the
real binary against fixture providers. Replace the flake's smoke-mock
check with a small packaged-cli check (version JSON, bash completions,
one mock quote), and cut scripts/live-smoke.sh from 672 to ~170 lines:
it now only runs shipped commands against live MSN/Yahoo/IDX and
validates output shape. Rewrite docs/SMOKE.md around the Nix gate and
the drift check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
hermes 2026-10-14 07:45:51 +00:00
commit 86c108f6eb
7 changed files with 188 additions and 771 deletions
Download patch file Download diff file Expand all files Collapse all files

4
.codex/skills/idx-release/SKILL.md
View file

@ -60,13 +60,13 @@ curl -fsSL https://crates.io/api/v1/crates/idx-cli
For a real release or release-hygiene fix, verify the repo state first. For a real release or release-hygiene fix, verify the repo state first.
Run the canonical gate (fmt, clippy, tests, release build, `cargo package` Run the canonical gate (fmt, clippy, tests, release build, `cargo package`
verification, and mock smoke): verification, and a packaged-binary check):
```bash ```bash
nix flake check -L nix flake check -L
``` ```
Run targeted smoke when the release changes user-facing CLI behavior. For Run `scripts/live-smoke.sh` when the release changes provider-facing behavior. For
ownership-sync changes, re-verify a clean-machine flow: ownership-sync changes, re-verify a clean-machine flow:
```bash ```bash

6
AGENTS.md
View file

@ -21,7 +21,7 @@ Docs:
- `docs/ARCHITECTURE.md`: providers, ownership ingest/discovery, error and output contract - `docs/ARCHITECTURE.md`: providers, ownership ingest/discovery, error and output contract
- `docs/CONVENTIONS.md`: types, naming, testing, and git rules - `docs/CONVENTIONS.md`: types, naming, testing, and git rules
- `docs/OWNERSHIP_SYNC.md` / `docs/OWNERSHIP_PUBLISH.md`: snapshot consumer contract / maintainer publishing - `docs/OWNERSHIP_SYNC.md` / `docs/OWNERSHIP_PUBLISH.md`: snapshot consumer contract / maintainer publishing
- `docs/SMOKE.md`: smoke runner and Nix check coverage - `docs/SMOKE.md`: Nix checks, test layout, and the live drift check
- `ROADMAP.md`: open work - `ROADMAP.md`: open work
## Rules ## Rules
@ -36,7 +36,7 @@ Docs:
## Verify ## Verify
```bash ```bash
nix flake check -L # canonical gate (fmt, clippy, tests, package, smoke) nix flake check -L # canonical gate (fmt, clippy, tests, package)
nix develop --command cargo test <name> # focused loop nix develop --command cargo test <name> # focused loop
scripts/live-smoke.sh --mode full # opt-in live providers scripts/live-smoke.sh # opt-in live provider drift check
``` ```

18
README.md
View file

@ -146,7 +146,7 @@ Precedence order:
- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md): providers, ownership ingest, and discovery - [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md): providers, ownership ingest, and discovery
- [docs/OWNERSHIP_SYNC.md](docs/OWNERSHIP_SYNC.md): snapshot manifest and sync rules - [docs/OWNERSHIP_SYNC.md](docs/OWNERSHIP_SYNC.md): snapshot manifest and sync rules
- [docs/OWNERSHIP_PUBLISH.md](docs/OWNERSHIP_PUBLISH.md): maintainer snapshot publishing - [docs/OWNERSHIP_PUBLISH.md](docs/OWNERSHIP_PUBLISH.md): maintainer snapshot publishing
- [docs/SMOKE.md](docs/SMOKE.md): smoke tests against live providers - [docs/SMOKE.md](docs/SMOKE.md): test layout and the live provider drift check
- [ROADMAP.md](ROADMAP.md): planned work - [ROADMAP.md](ROADMAP.md): planned work
## Development ## Development
@ -157,10 +157,10 @@ The canonical local and CI verification command is:
nix flake check -L nix flake check -L
``` ```
It covers Rust formatting, Clippy, tests, crate package verification, release It covers Rust formatting, Clippy, unit and CLI integration tests, crate
builds, offline CLI smoke, publisher regressions, installer tests with `sh` and package verification, the packaged binary, publisher regressions, installer
`dash`, and ShellCheck for installer/smoke scripts. The checks use pinned tools and cached Nix tests with `sh` and `dash`, and ShellCheck. The checks use pinned tools and
outputs; an unchanged successful check is reused rather than rerun. cached Nix outputs; an unchanged successful check is reused rather than rerun.
For a focused edit/debug loop, use the same pinned toolchain: For a focused edit/debug loop, use the same pinned toolchain:
@ -172,10 +172,10 @@ Configured Prek hooks keep whitespace/conflict checks at pre-commit and run
`nix flake check -L` at pre-push. There is no separate full Cargo test/lint gate `nix flake check -L` at pre-push. There is no separate full Cargo test/lint gate
to repeat after a successful flake check. to repeat after a successful flake check.
CI additionally checks the installer with native macOS tools for portability. CI additionally checks the installer with native macOS tools. The live
Live-provider smoke remains opt-in; see [docs/SMOKE.md](docs/SMOKE.md) for check provider drift check (`scripts/live-smoke.sh`) is opt-in; see
selection and caching details. The npm distribution harness remains available [docs/SMOKE.md](docs/SMOKE.md). The npm wrapper has its own harness,
separately as `scripts/npm-smoke.sh`. `scripts/npm-smoke.sh`.
## License ## License

2
docs/CONVENTIONS.md
View file

@ -119,7 +119,7 @@ let conn = Connection::open(path)
- **Unit tests:** retain precise parser, normalization, calculation, and boundary regressions; avoid pinning incidental wording, enum counts, or internal wiring. - **Unit tests:** retain precise parser, normalization, calculation, and boundary regressions; avoid pinning incidental wording, enum counts, or internal wiring.
- **Integration tests:** prefer CLI workflows, real temporary/in-memory SQLite, and deterministic local provider fixtures for user-visible contracts and state transitions. - **Integration tests:** prefer CLI workflows, real temporary/in-memory SQLite, and deterministic local provider fixtures for user-visible contracts and state transitions.
- **Output contracts:** parse JSON from stdout on success and stderr on failure; errors must leave stdout empty. Assert meaningful values, types, and stable error codes rather than substrings or diagnostic wording. - **Output contracts:** parse JSON from stdout on success and stderr on failure; errors must leave stdout empty. Assert meaningful values, types, and stable error codes rather than substrings or diagnostic wording.
- **Stateful workflows:** compare cached data across offline/failure transitions, and verify failed snapshot updates preserve both database bytes and query results. Keep packaged-binary smoke representative; put exhaustive boundaries in integration tests. - **Stateful workflows:** compare cached data across offline/failure transitions, and verify failed snapshot updates preserve both database bytes and query results. Put behavioral coverage in `tests/cli.rs`, not shell smoke.
- **Fixtures:** `tests/fixtures/*.json` — real API responses, sanitized - **Fixtures:** `tests/fixtures/*.json` — real API responses, sanitized
- **No live API calls in CI** — `IDX_USE_MOCK_PROVIDER=1` - **No live API calls in CI** — `IDX_USE_MOCK_PROVIDER=1`
- **Test naming:** `test_<function>_<scenario>` (e.g., `test_parse_id_number_with_dots`) - **Test naming:** `test_<function>_<scenario>` (e.g., `test_parse_id_number_with_dots`)

154
docs/SMOKE.md
View file

@ -1,135 +1,55 @@
# Smoke Checks # Testing and Smoke Checks
Use `scripts/live-smoke.sh` as the reusable smoke runner for shipped CLI surfaces. ## Automated gate
The script: `nix flake check -L` is the canonical local, pre-push, and CI gate. It runs
- builds `target/debug/idx` once by default offline with pinned tools:
- isolates config, cache, and data under `tmp/live-smoke/<timestamp>/`
- records separate `.stdout` and `.stderr` files per case under `tmp/live-smoke/<timestamp>/logs/`, plus `.validation` diagnostics for JSON checks
- supports fast mock-backed runs and slower live-network passes; `jq` is required and provided by `nix develop`
## Common Runs | Check | Covers |
| --- | --- |
| `fmt`, `clippy` | formatting and lints (`--all-targets`) |
| `test` | unit/module tests and `tests/cli.rs` CLI integration tests |
| `default`, `package` | release build and `cargo package` verification |
| `packaged-cli` | the wrapped Nix binary starts, emits JSON, and generates completions |
| `ownership-publisher`, `publisher-test` | packaged publisher entry point and its regression scenarios |
| `install-script-sh`, `install-script-dash`, `shellcheck` | installer regressions and shell lint |
```bash `tests/cli.rs` is the behavioral suite: it runs the real `idx` binary against
scripts/live-smoke.sh fixture-backed mock providers (`IDX_USE_MOCK_PROVIDER=1`) and covers output
scripts/live-smoke.sh --mode full contracts, cache/offline transitions, provider routing, error codes, ownership
scripts/live-smoke.sh --mode mock imports and queries, and snapshot sync.
scripts/live-smoke.sh --group live-table --group live-json
scripts/live-smoke.sh --group live-nonfinite
scripts/live-smoke.sh --group cache --symbol BBRI
scripts/live-smoke.sh --dry-run --mode full
scripts/live-smoke.sh --bin ./tmp/release-install/bin/idx --no-build --mode mock
scripts/audit-msn-fundamentals.sh --tickers BUMI,ADRO,AIMS
```
## Nix CI Checks For a focused loop, run one check or filter Cargo tests:
`nix flake check -L` is the canonical local, pre-push, and Linux CI gate. Its
eleven checks cover:
- Rust formatting, Clippy (`--all-targets`), and all unit/CLI tests
- the optimized application build and verified crate packaging
- the packaged ownership publisher's `--help` entry point
- the offline mock CLI smoke matrix
- publisher regression scenarios, including the SQLite artifact builder
- installer regressions under `sh` and `dash`, plus installer and smoke-runner ShellCheck
All check tools are pinned by `flake.lock`. Installer fixtures use resolved
shell paths, and generated publisher stubs use the running Bash interpreter,
so the regression suites also work inside the Nix sandbox.
Crane shares dev-profile dependency artifacts between tests, Clippy, and
`cargo package` verification; the shipped application uses a separate release
dependency cache. Package verification still builds the packaged crate.
Integration-test edits do not invalidate the application build, and script
checks consume only their relevant source files. README/license edits affect
the packaging check, not the Rust test/application derivations.
CI cache keys include platform, dependency/toolchain inputs, and checked source
contents. Source changes can therefore save new outputs; restore prefixes first
reuse the same dependency/toolchain cache, then another cache for the platform.
The GitHub cache is opportunistic and subject to eviction, not a shared public
binary cache. Obsolete PR runs are cancelled; main-branch runs are not.
Use individual checks when investigating a failure (replace the system as
needed), or targeted Cargo commands inside the pinned development shell:
```bash ```bash
nix build .#checks.x86_64-linux.publisher-test -L nix build .#checks.x86_64-linux.publisher-test -L
nix develop --command cargo test full_screener nix develop --command cargo test full_screener
``` ```
The native macOS installer job remains separate to exercise BSD/macOS tools. CI cache keys include platform, dependency/toolchain inputs, and checked source
Live-network smoke is deliberately outside the sandboxed offline gate. contents. The native macOS installer job runs separately to exercise BSD tools.
For a live full-list screener check, use an isolated config/cache and run: ## Live drift check
Fixtures cannot tell when MSN, Yahoo, or IDX change their responses.
`scripts/live-smoke.sh` runs shipped commands against the real upstreams and
validates each JSON output shape. It is opt-in and never part of the gate.
```bash ```bash
idx -o json stocks screen --region us --filter top-performers --limit 600 nix develop --command scripts/live-smoke.sh # stocks + ownership
idx --offline -o json stocks screen --region us --filter top-performers --limit 3 nix develop --command scripts/live-smoke.sh --group stocks --symbol BBRI
idx --offline -o json stocks screen --region us --filter top-performers --limit 25 nix develop --command scripts/live-smoke.sh --bin ./result/bin/idx
scripts/live-smoke.sh --dry-run
``` ```
The first command should not stop at the old 500-row cap when MSN reports more - `stocks`: every `stocks` command for one ticker, plus `compare BUMI,ADRO,AIMS`,
candidates. The two offline results must match prefixes of the same cached full whose MSN key ratios contain non-finite values.
list. Provider counts and prices vary; live HTTP failures are not offline-test - `ownership`: `discover`, then imports the newest supported report into a temp
failures. DB and lists `releases`. IDX sits behind Cloudflare and returns `403` to many
datacenter IPs, including GitHub-hosted runners; run it from a network IDX accepts.
## Coverage Boundaries Each run isolates config, cache, and data under `tmp/live-smoke/<timestamp>/`
and keeps per-case `.stdout`, `.stderr`, and `.validation` logs there.
The mock preset has 43 cases: 7 general, 19 stock, 9 cache, 2 routing, 4 error, `scripts/audit-msn-fundamentals.sh` is a heavier sweep of valuation output
and 2 empty/unsupported ownership checks. This is a packaged-binary safety net, across the whole IDX MSN symbol map, for provider-health audits.
not a claim that every important behavior is covered by shell smoke.
- All fifteen stock commands run in JSON mode with structural and fixture-value
assertions. Four representative table renders check displayed data.
- JSON checks require exactly one document in the correct stream. Failed
commands must leave stdout empty and expose the expected stable error code.
- Cache scenarios use zero TTL, force provider failures, and compare parsed
offline/stale results with the warmed data; diagnostics must stay on stderr.
- Redundant table launches, duplicate routing cases, and repeated setup checks
are omitted rather than counted as extra correctness coverage.
The CLI integration suite carries deeper workflows under the same Nix gate:
cache misses, provider isolation, filtered reports, non-finite values,
XLSX/ZIP import and ownership queries, snapshot install/no-op/force, and rejected
updates preserving database bytes and query results.
Screener regressions generate 503 distinct candidates, with ranking leaders
beyond the first 500. They exercise top/worst performers, volume, and market-cap
ordering, a small first request followed by complete offline reads, and rejected
missing/incomplete totals without cache poisoning. The raw fixture override
`IDX_MOCK_MSN_SCREENER_FIXTURE` is honored only with `IDX_USE_MOCK_PROVIDER`;
responses honor the requested batch size so the complete-fetch path is exercised.
Mocks do not verify current upstream availability, authentication, or response
drift. Live provider checks remain a separate, opt-in operation.
## Modes
- `live` runs the default real-network baseline: `general`, `live-table`, and `ownership`
- `mock` runs the deterministic baseline: `general`, `mock`, `cache`, `routing`, `errors`, and `ownership`
- `full` runs every group, including live JSON output checks and ownership-import verification
## Groups
- `general`: `version`, `completions`, `config`, and `cache` command basics
- `live-table`: all shipped `stocks` commands in live table mode
- `live-json`: all shipped `stocks` commands in live JSON mode
- `mock`: all shipped `stocks` commands against the mock provider in JSON mode, plus representative quote/history/fundamental/financials table renders
- `cache`: exact data equality across warm, `--offline`, and stale-cache fallback reads for quote, technical, and MSN `profile`
- `routing`: Yahoo/MSN provider routing plus explicit MSN history behavior
- `errors`: JSON error contract and invalid flag/input checks
- `live-nonfinite`: opt-in live MSN fundamentals checks for known non-finite ticker payloads (`BUMI`, `ADRO`, `AIMS`)
- `ownership`: safe ownership smoke checks that do not require imported ownership data
- `ownership-import`: live discovery/import hardening checks for supported `above1` import plus expected unsupported legacy-family failures
## Notes
- The runner forces `IDX_OUTPUT=table` so table cases stay stable; JSON checks pass `-o json` explicitly.
- Cache-group warm cases clear the smoke cache first so warm/offline/stale sequences start clean.
- Use `--bin <path> --no-build` to validate an installed binary instead of `target/debug/idx`.
- `ownership-import` and `live-nonfinite` are opt-in (explicit `--group` or `--mode full`). Known non-finite repro tickers as of `2026-04-13`: `BUMI`, `ADRO`, `AIMS`.
- The baseline `ownership` group needs no imported data; `ownership sync` is covered by fixture-backed CLI tests instead.
- `scripts/audit-msn-fundamentals.sh` is a heavier provider-health sweep over the IDX MSN symbol map, not a baseline check.
- When a case fails, inspect its `.stdout`, `.stderr`, and `.validation` files under `tmp/live-smoke/.../logs/`.

16
flake.nix
View file

@ -189,19 +189,19 @@
doInstallCargoArtifacts = false; doInstallCargoArtifacts = false;
installPhaseCommand = "touch $out"; installPhaseCommand = "touch $out";
}); });
smoke-mock = pkgs.runCommand "idx-smoke-mock" # Sanity check of the packaged (wrapped) binary; CLI behavior is
{ nativeBuildInputs = with pkgs; [ bash coreutils diffutils findutils gnugrep gnused gawk jq ]; } # covered by tests/cli.rs.
packaged-cli = pkgs.runCommand "idx-packaged-cli"
{ nativeBuildInputs = with pkgs; [ bash jq ]; }
'' ''
mkdir -p scripts
cp ${./scripts/live-smoke.sh} scripts/live-smoke.sh
chmod u+w scripts/live-smoke.sh
patchShebangs scripts/live-smoke.sh
# The mock provider reads fixtures relative to the working dir. # The mock provider reads fixtures relative to the working dir.
mkdir -p tests mkdir -p tests
cp -r ${./tests/fixtures} tests/fixtures cp -r ${./tests/fixtures} tests/fixtures
export HOME="$TMPDIR" export HOME="$TMPDIR"
${idxPackage}/bin/idx version ${idxPackage}/bin/idx -o json version | jq -e '.version | type == "string"'
bash scripts/live-smoke.sh --bin ${idxPackage}/bin/idx --no-build --mode mock ${idxPackage}/bin/idx completions bash | bash -n
IDX_USE_MOCK_PROVIDER=1 ${idxPackage}/bin/idx -o json stocks quote BBCA \
| jq -e '.[0].price | type == "number"'
touch "$out" touch "$out"
''; '';
install-script-sh = installerCheck "sh"; install-script-sh = installerCheck "sh";

755
scripts/live-smoke.sh
View file

@ -1,672 +1,169 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Live provider drift check: runs shipped commands against real MSN, Yahoo,
# and IDX endpoints and validates the JSON output shape. Deterministic CLI
# behavior is covered by tests/cli.rs; this only answers "do the upstreams
# still look the way our parsers expect?"
set -u set -u
set -o pipefail set -o pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
MODE="live"
BIN_PATH=""
WORKDIR=""
SYMBOL="BBCA" SYMBOL="BBCA"
COMPARE_SYMBOLS="BBCA,BBRI" BIN_PATH=""
NEWS_LIMIT="5"
SCREEN_LIMIT="10"
DRY_RUN=0 DRY_RUN=0
BUILD=1 declare -a GROUPS_SELECTED=()
declare -a GROUP_FILTERS=()
declare -a CASES=()
declare -a FAILURES=() declare -a FAILURES=()
TOTAL=0
TOTAL_CASES=0 PASSED=0
PASSED_CASES=0
FAILED_CASES=0
SEP=$'\x1f'
CONFIG_HOME=""
CACHE_HOME=""
DATA_HOME=""
OWNERSHIP_DB=""
LOG_DIR=""
OWNERSHIP_ABOVE1_URL=""
OWNERSHIP_ABOVE5_URL=""
OWNERSHIP_INVESTOR_TYPE_URL=""
declare -a BASE_ENV=()
usage() { usage() {
cat <<'EOF' cat <<'EOF'
Usage: scripts/live-smoke.sh [options] Usage: scripts/live-smoke.sh [options]
Reusable smoke runner for shipped idx-cli command surfaces.
Options: Options:
--mode <live|mock|full> Preset case selection. Default: live --group <stocks|ownership> Restrict to a group. Repeatable. Default: both
--group <name> Restrict to one or more groups. Repeatable. --symbol <ticker> Primary ticker. Default: BBCA
--symbol <ticker> Primary ticker for stock cases. Default: BBCA --bin <path> Use an existing idx binary (skips cargo build)
--compare-symbols <list> Compare symbols. Default: BBCA,BBRI --dry-run Print cases without running them
--news-limit <n> News limit. Default: 5 --help Show this help
--screen-limit <n> Screener limit. Default: 10
--workdir <path> Artifact root. Default: tmp/live-smoke/<timestamp>
--bin <path> Use an existing idx binary instead of target/debug/idx
--no-build Skip cargo build
--dry-run Print selected cases without running them
--help Show this help
Groups:
general version/completions/config/cache basics
live-table live network stock commands in table mode
live-json live network stock commands in JSON mode
mock mock-backed stock JSON contracts + representative table checks
cache deterministic cache/offline/stale-cache checks
routing provider routing and explicit unsupported checks
errors JSON error contract and invalid-flag checks
live-nonfinite opt-in live MSN fundamentals checks for known non-finite tickers
ownership ownership commands that are safe without imported data
ownership-import live ownership discovery/import hardening checks
Examples:
scripts/live-smoke.sh
scripts/live-smoke.sh --mode full
scripts/live-smoke.sh --mode mock --group cache
scripts/live-smoke.sh --group live-table --group live-json --symbol BBRI
scripts/live-smoke.sh --group live-nonfinite
EOF EOF
} }
add_group_filters() { while [[ $# -gt 0 ]]; do
local raw="$1" case "$1" in
local part --group) GROUPS_SELECTED+=("${2:-}"); shift 2 ;;
local old_ifs="$IFS" --symbol) SYMBOL="${2:-}"; shift 2 ;;
IFS=',' --bin) BIN_PATH="${2:-}"; shift 2 ;;
for part in $raw; do --dry-run) DRY_RUN=1; shift ;;
if [[ -n "$part" ]]; then --help|-h) usage; exit 0 ;;
GROUP_FILTERS+=("$part") *) echo "unknown argument: $1" >&2; usage >&2; exit 2 ;;
fi
done
IFS="$old_ifs"
}
mode_allows_group() {
local group="$1"
case "$MODE" in
live)
case "$group" in
general|live-table|ownership) return 0 ;;
*) return 1 ;;
esac
;;
mock)
case "$group" in
general|mock|cache|routing|errors|ownership) return 0 ;;
*) return 1 ;;
esac
;;
full)
return 0
;;
*)
return 1
;;
esac esac
} done
[[ ${#GROUPS_SELECTED[@]} -eq 0 ]] && GROUPS_SELECTED=(stocks ownership)
group_requested() { for group in "${GROUPS_SELECTED[@]}"; do
local group="$1" case "$group" in
local selected stocks|ownership) ;;
*) echo "invalid --group '$group' (expected stocks or ownership)" >&2; exit 2 ;;
esac
done
if [[ ${#GROUP_FILTERS[@]} -eq 0 ]]; then selected() {
return 0 local group
fi for group in "${GROUPS_SELECTED[@]}"; do
[[ "$group" == "$1" ]] && return 0
for selected in "${GROUP_FILTERS[@]}"; do
if [[ "$selected" == "$group" ]]; then
return 0
fi
done done
return 1 return 1
} }
should_run_group() { WORKDIR="$ROOT_DIR/tmp/live-smoke/$(date +%Y%m%d-%H%M%S)"
local group="$1" LOG_DIR="$WORKDIR/logs"
if [[ ${#GROUP_FILTERS[@]} -gt 0 ]]; then BASE_ENV=(
group_requested "$group" "XDG_CONFIG_HOME=$WORKDIR/config"
return "XDG_CACHE_HOME=$WORKDIR/cache"
fi "XDG_DATA_HOME=$WORKDIR/data"
"IDX_PROVIDER=msn"
"IDX_HISTORY_PROVIDER=auto"
"IDX_NO_COLOR=1"
)
mode_allows_group "$group" # jq filter describing the JSON shape each command must keep.
} contract() {
case "$1" in
add_case() { "stocks quote "*|"stocks screen"*)
CASES+=("$1$SEP$2$SEP$3$SEP$4$SEP$5$SEP$6$SEP${7:-}") echo 'type == "array" and length > 0 and all(.[]; (.symbol | type == "string") and (.price | type == "number"))' ;;
} "stocks history "*)
echo 'type == "array" and length > 0 and all(.[]; (.date | type == "string") and (.close | type == "number"))' ;;
register_cases() { "stocks technical "*)
local live_env="IDX_PROVIDER=msn IDX_HISTORY_PROVIDER=auto" echo '(.current_price | type == "number") and (.signals.overall | type == "string")' ;;
local mock_env="IDX_PROVIDER=msn IDX_HISTORY_PROVIDER=auto IDX_USE_MOCK_PROVIDER=1" "stocks growth "*|"stocks valuation "*|"stocks risk "*)
local yahoo_mock_env="IDX_PROVIDER=yahoo IDX_HISTORY_PROVIDER=auto IDX_USE_MOCK_PROVIDER=1" echo '.overall_signal | type == "string"' ;;
local quote_zero_ttl_env="IDX_PROVIDER=msn IDX_USE_MOCK_PROVIDER=1 IDX_CACHE_QUOTE_TTL=0" "stocks fundamental "*)
local profile_zero_ttl_env="IDX_PROVIDER=msn IDX_USE_MOCK_PROVIDER=1 IDX_CACHE_FUNDAMENTAL_TTL=0" echo '(.growth.overall_signal | type == "string") and (.valuation.overall_signal | type == "string") and (.risk.overall_signal | type == "string")' ;;
local live_symbol="$SYMBOL" "stocks compare "*)
local compare_args="$COMPARE_SYMBOLS" echo 'type == "array" and length > 0 and all(.[]; (.symbol | type == "string") and (.valuation | type == "object"))' ;;
local history_args="stocks history $live_symbol --period 3mo" "stocks profile "*)
local screen_args="stocks screen --filter top-performers --limit $SCREEN_LIMIT" echo '(.symbol | type == "string") and has("long_name") and (.officers | type == "array")' ;;
local news_args="stocks news $live_symbol --limit $NEWS_LIMIT" "stocks financials "*)
echo '(.instrument.symbol | type == "string") and has("income_statement") and has("balance_sheet") and has("cash_flow")' ;;
add_case "general" "version" "0" "" "-o json version" "" '.version | type == "string" and test("^[0-9]+\\.[0-9]+\\.[0-9]+")' "stocks earnings "*)
add_case "general" "completions-bash" "0" "" "completions bash" "" echo '(.history | type == "array") and (.forecast | type == "array")' ;;
add_case "general" "config-set-output-json" "0" "" "config set general.output json" "" "stocks sentiment "*)
add_case "general" "config-get-output" "0" "" "config get general.output" "json" echo '.statistics | type == "array"' ;;
add_case "general" "config-set-output-table" "0" "" "config set general.output table" "" "stocks insights "*)
add_case "general" "cache-info" "0" "" "-o json cache info" "" '.files | type == "number" and . >= 0' echo '(.highlights | type == "array") and (.risks | type == "array")' ;;
add_case "general" "cache-clear" "0" "" "-o json cache clear" "" '.removed | type == "number" and . >= 0' "stocks news "*)
echo 'type == "array" and all(.[]; (.title | type == "string") and (.url | type == "string"))' ;;
add_case "live-table" "quote" "0" "$live_env" "stocks quote $live_symbol" "" "ownership discover"*)
add_case "live-table" "history" "0" "$live_env" "$history_args" "" echo 'type == "array" and length > 0 and (.[0].pdf_url | type == "string")' ;;
add_case "live-table" "technical" "0" "$live_env" "stocks technical $live_symbol" "" "ownership releases")
add_case "live-table" "growth" "0" "$live_env" "stocks growth $live_symbol" "" echo 'type == "array" and length == 1' ;;
add_case "live-table" "valuation" "0" "$live_env" "stocks valuation $live_symbol" ""
add_case "live-table" "risk" "0" "$live_env" "stocks risk $live_symbol" ""
add_case "live-table" "fundamental" "0" "$live_env" "stocks fundamental $live_symbol" ""
add_case "live-table" "compare" "0" "$live_env" "stocks compare $compare_args" ""
add_case "live-table" "profile" "0" "$live_env" "stocks profile $live_symbol" ""
add_case "live-table" "financials" "0" "$live_env" "stocks financials $live_symbol" ""
add_case "live-table" "earnings" "0" "$live_env" "stocks earnings $live_symbol" ""
add_case "live-table" "sentiment" "0" "$live_env" "stocks sentiment $live_symbol" ""
add_case "live-table" "insights" "0" "$live_env" "stocks insights $live_symbol" ""
add_case "live-table" "news" "0" "$live_env" "$news_args" ""
add_case "live-table" "screen" "0" "$live_env" "$screen_args" ""
add_case "live-json" "quote" "0" "$live_env" "-o json stocks quote $live_symbol" ""
add_case "live-json" "history" "0" "$live_env" "-o json $history_args" ""
add_case "live-json" "technical" "0" "$live_env" "-o json stocks technical $live_symbol" ""
add_case "live-json" "growth" "0" "$live_env" "-o json stocks growth $live_symbol" ""
add_case "live-json" "valuation" "0" "$live_env" "-o json stocks valuation $live_symbol" ""
add_case "live-json" "risk" "0" "$live_env" "-o json stocks risk $live_symbol" ""
add_case "live-json" "fundamental" "0" "$live_env" "-o json stocks fundamental $live_symbol" ""
add_case "live-json" "compare" "0" "$live_env" "-o json stocks compare $compare_args" ""
add_case "live-json" "profile" "0" "$live_env" "-o json stocks profile $live_symbol" ""
add_case "live-json" "financials" "0" "$live_env" "-o json stocks financials $live_symbol" ""
add_case "live-json" "earnings" "0" "$live_env" "-o json stocks earnings $live_symbol" ""
add_case "live-json" "sentiment" "0" "$live_env" "-o json stocks sentiment $live_symbol" ""
add_case "live-json" "insights" "0" "$live_env" "-o json stocks insights $live_symbol" ""
add_case "live-json" "news" "0" "$live_env" "-o json $news_args" ""
add_case "live-json" "screen" "0" "$live_env" "-o json $screen_args" ""
add_case "live-nonfinite" "valuation-bumi" "0" "$live_env" "-o json stocks valuation BUMI" ""
add_case "live-nonfinite" "valuation-adro" "0" "$live_env" "-o json stocks valuation ADRO" ""
add_case "live-nonfinite" "valuation-aims" "0" "$live_env" "-o json stocks valuation AIMS" ""
add_case "live-nonfinite" "compare-bad-tickers" "0" "$live_env" "-o json stocks compare BUMI,ADRO,AIMS" "" 'any(.[]; .symbol == "BUMI.JK")'
add_case "mock" "quote-table" "0" "$mock_env" "stocks quote $live_symbol" "9,875"
add_case "mock" "quote-json" "0" "$mock_env" "-o json stocks quote $live_symbol" "" '.[0].price == 9875 and .[0].change == 117'
add_case "mock" "history-table" "0" "$mock_env" "$history_args" "2024-03-03"
add_case "mock" "history-json" "0" "$mock_env" "-o json $history_args" "" '.[-1].close == 9925'
add_case "mock" "technical-json" "0" "$mock_env" "-o json stocks technical $live_symbol" "" '.current_price == 9925 and .sma200 == null'
add_case "mock" "growth-json" "0" "$mock_env" "-o json stocks growth $live_symbol" "" '.revenue_growth_pct == 8.1 and .earnings_growth_pct == 12.1'
add_case "mock" "valuation-json" "0" "$mock_env" "-o json stocks valuation $live_symbol" "" '.pe_trailing == 25.4 and .roe_pct == 19.8'
add_case "mock" "risk-json" "0" "$mock_env" "-o json stocks risk $live_symbol" "" '.debt_to_equity == 0.42 and .roa_pct == 3.1'
add_case "mock" "fundamental-table" "0" "$mock_env" "stocks fundamental $live_symbol" "25.40"
add_case "mock" "fundamental-json" "0" "$mock_env" "-o json stocks fundamental $live_symbol" "" '.growth.revenue_growth_pct == 8.1 and .valuation.pe_trailing == 25.4 and .risk.debt_to_equity == 0.42'
add_case "mock" "compare-json" "0" "$mock_env" "-o json stocks compare $compare_args" "" 'length > 0 and all(.[]; .growth.revenue_growth_pct == 8.1 and .valuation.pe_trailing == 25.4)'
add_case "mock" "profile-json" "0" "$mock_env" "-o json stocks profile $live_symbol" "" '.country == "Indonesia" and .employees == 27937'
add_case "mock" "financials-table" "0" "$mock_env" "stocks financials $live_symbol" "400,000,000"
add_case "mock" "financials-json" "0" "$mock_env" "-o json stocks financials $live_symbol" "" '.income_statement.values.netIncome == 400000000 and .cash_flow.values.operatingCashFlow == 600000000'
add_case "mock" "earnings-json" "0" "$mock_env" "-o json stocks earnings $live_symbol" "" '.eps_last_year == 1200 and .forecast[0].eps_forecast == 1300 and .history[0].eps_actual == 1250'
add_case "mock" "sentiment-json" "0" "$mock_env" "-o json stocks sentiment $live_symbol" "" '.statistics[0] == {"time_range":"1D","bullish":10,"bearish":2,"neutral":3}'
add_case "mock" "insights-json" "0" "$mock_env" "-o json stocks insights $live_symbol" "" '.last_updated == "2026-03-26T04:14:57.9197955Z"'
add_case "mock" "news-json" "0" "$mock_env" "-o json $news_args" "" "map(.id) == ([\"news-1\"][:$NEWS_LIMIT])"
add_case "mock" "screen-json" "0" "$mock_env" "-o json $screen_args" "" "map(.symbol) == ([\"BBRI.JK\", \"BBCA.JK\"][:$SCREEN_LIMIT])"
add_case "cache" "quote-warm" "0" "$quote_zero_ttl_env" "-o json stocks quote $live_symbol" ""
add_case "cache" "quote-offline" "0" "$quote_zero_ttl_env IDX_MOCK_ERROR=1" "-o json --offline stocks quote $live_symbol" ""
add_case "cache" "quote-stale-fallback" "0" "$quote_zero_ttl_env IDX_MOCK_ERROR=1" "-o json stocks quote $live_symbol" ""
add_case "cache" "technical-warm" "0" "$quote_zero_ttl_env" "-o json stocks technical $live_symbol" ""
add_case "cache" "technical-offline" "0" "$quote_zero_ttl_env IDX_MOCK_ERROR=1" "-o json --offline stocks technical $live_symbol" ""
add_case "cache" "technical-stale-fallback" "0" "$quote_zero_ttl_env IDX_MOCK_ERROR=1" "-o json stocks technical $live_symbol" ""
add_case "cache" "profile-warm" "0" "$profile_zero_ttl_env" "-o json stocks profile $live_symbol" ""
add_case "cache" "profile-offline" "0" "$profile_zero_ttl_env IDX_MOCK_ERROR=1" "-o json --offline stocks profile $live_symbol" ""
add_case "cache" "profile-stale-fallback" "0" "$profile_zero_ttl_env IDX_MOCK_ERROR=1" "-o json stocks profile $live_symbol" ""
add_case "routing" "yahoo-quote" "0" "$yahoo_mock_env" "-o json --no-cache stocks quote $live_symbol" "" '.[0].price == 9875'
add_case "routing" "explicit-msn-history" "0" "$mock_env" "-o json --no-cache stocks history $live_symbol --period 3mo --history-provider msn" "" '.[0].close == 8000 and all(.[]; .volume == 0 and .open == .close and .high == .close and .low == .close)'
add_case "errors" "invalid-provider-json" "1" "IDX_PROVIDER=bogus" "-o json version" "" '.code == "CONFIGERROR"'
add_case "errors" "profile-provider-gate-json" "1" "IDX_PROVIDER=yahoo" "-o json stocks profile $live_symbol" "" '.code == "UNSUPPORTED"'
add_case "errors" "invalid-screen-region-json" "1" "$mock_env" "-o json stocks screen --region eu" "" '.code == "INVALIDINPUT"'
add_case "errors" "offline-no-cache" "1" "IDX_USE_MOCK_PROVIDER=1" "-o json --offline --no-cache stocks quote $live_symbol" "" '.code == "INVALIDINPUT"'
add_case "ownership" "releases-empty" "0" "" "-o json ownership releases" "" '. == []'
add_case "ownership" "fetch-bing-unsupported" "1" "" "-o json ownership import --fetch-bing $live_symbol" "" '.code == "UNSUPPORTED"'
add_case "ownership-import" "discover-default-above1" "0" "" "ownership discover --limit 1" "supported"
add_case "ownership-import" "import-supported-above1" "0" "" "ownership import --url $OWNERSHIP_ABOVE1_URL" "Imported "
add_case "ownership-import" "releases-after-import" "0" "" "ownership releases" "$OWNERSHIP_ABOVE1_URL"
add_case "ownership-import" "import-legacy-above5" "1" "" "ownership import --url $OWNERSHIP_ABOVE5_URL" "legacy IDX \`above5\` ownership PDFs are not supported for import"
add_case "ownership-import" "import-legacy-investor-type" "1" "" "ownership import --url $OWNERSHIP_INVESTOR_TYPE_URL" "legacy IDX \`investor-type\` ownership PDFs are not supported for import"
}
parse_args() {
while [[ $# -gt 0 ]]; do
case "$1" in
--mode)
MODE="${2:-}"
shift 2
;;
--group)
add_group_filters "${2:-}"
shift 2
;;
--symbol)
SYMBOL="${2:-}"
shift 2
;;
--compare-symbols)
COMPARE_SYMBOLS="${2:-}"
shift 2
;;
--news-limit)
NEWS_LIMIT="${2:-}"
shift 2
;;
--screen-limit)
SCREEN_LIMIT="${2:-}"
shift 2
;;
--workdir)
WORKDIR="${2:-}"
shift 2
;;
--bin)
BIN_PATH="${2:-}"
BUILD=0
shift 2
;;
--no-build)
BUILD=0
shift
;;
--dry-run)
DRY_RUN=1
shift
;;
--help|-h)
usage
exit 0
;;
*)
echo "unknown argument: $1" >&2
usage >&2
exit 2
;;
esac
done
case "$MODE" in
live|mock|full) ;;
*) *)
echo "invalid --mode '$MODE' (expected live, mock, or full)" >&2 echo 'true' ;;
exit 2
;;
esac esac
} }
prepare_paths() { run_case() {
if [[ -z "$WORKDIR" ]]; then local name="$1" cmd="$2" filter log
WORKDIR="$ROOT_DIR/tmp/live-smoke/$(date +%Y%m%d-%H%M%S)" local -a args
TOTAL=$((TOTAL + 1))
printf '[%02d] %s\n' "$TOTAL" "$name"
if (( DRY_RUN )); then
printf ' idx -o json %s\n' "$cmd"
return
fi fi
CONFIG_HOME="$WORKDIR/config" log="$LOG_DIR/$(printf '%02d' "$TOTAL")_$name"
CACHE_HOME="$WORKDIR/cache" read -r -a args <<< "$cmd"
DATA_HOME="$WORKDIR/data" if ! env "${BASE_ENV[@]}" "$BIN_PATH" -o json "${args[@]}" >"$log.stdout" 2>"$log.stderr"; then
OWNERSHIP_DB="$WORKDIR/ownership/ownership.db" FAILURES+=("$name: non-zero exit (see $log.stderr)")
LOG_DIR="$WORKDIR/logs" echo " FAIL exit"
return
BASE_ENV=(
"XDG_CONFIG_HOME=$CONFIG_HOME"
"XDG_CACHE_HOME=$CACHE_HOME"
"XDG_DATA_HOME=$DATA_HOME"
"IDX_OUTPUT=table"
"IDX_NO_COLOR=1"
)
if (( DRY_RUN == 0 )); then
mkdir -p "$CONFIG_HOME" "$CACHE_HOME" "$DATA_HOME" "$(dirname "$OWNERSHIP_DB")" "$LOG_DIR"
fi fi
filter="$(contract "$cmd")"
if ! jq -e -s "length == 1 and (.[0] | $filter)" "$log.stdout" >/dev/null 2>"$log.validation"; then
FAILURES+=("$name: output shape changed (see $log.stdout)")
echo " FAIL shape"
return
fi
PASSED=$((PASSED + 1))
echo " PASS"
} }
build_binary() { if (( ! DRY_RUN )); then
local build_log command -v jq >/dev/null 2>&1 || { echo "jq is required; run inside nix develop" >&2; exit 1; }
local stale_input mkdir -p "$LOG_DIR"
if [[ -z "$BIN_PATH" ]]; then if [[ -z "$BIN_PATH" ]]; then
BIN_PATH="$ROOT_DIR/target/debug/idx" BIN_PATH="$ROOT_DIR/target/debug/idx"
(cd "$ROOT_DIR" && cargo build --quiet --bin idx) || { echo "cargo build failed" >&2; exit 1; }
fi fi
env "${BASE_ENV[@]}" "$BIN_PATH" config set ownership.db_path "$WORKDIR/ownership.db" >/dev/null
fi
if (( DRY_RUN )); then if selected stocks; then
return 0 for cmd in quote technical growth valuation risk fundamental profile financials earnings sentiment insights; do
fi run_case "$cmd" "stocks $cmd $SYMBOL"
if (( BUILD )); then
if ! command -v cargo >/dev/null 2>&1; then
if [[ -x "$BIN_PATH" ]]; then
printf 'info: cargo not found; reusing existing binary at %s\n' "$BIN_PATH"
BUILD=0
else
echo "cargo not found; run inside nix develop or pass --bin/--no-build with an existing idx binary" >&2
exit 1
fi
fi
fi
if (( BUILD == 0 )) && [[ "$BIN_PATH" == "$ROOT_DIR/target/debug/idx" && -x "$BIN_PATH" ]]; then
stale_input="$(
find \
"$ROOT_DIR/src" \
"$ROOT_DIR/tests" \
"$ROOT_DIR/Cargo.toml" \
"$ROOT_DIR/Cargo.lock" \
"$ROOT_DIR/scripts/live-smoke.sh" \
-type f \
-newer "$BIN_PATH" \
-print \
-quit \
2>/dev/null
)"
if [[ -n "$stale_input" ]]; then
echo "refusing to run smoke checks against stale $BIN_PATH; newer input detected at $stale_input" >&2
echo "rebuild first or omit --no-build so the runner refreshes target/debug/idx" >&2
exit 1
fi
fi
if (( BUILD )); then
build_log="$LOG_DIR/build.log"
if ! (
cd "$ROOT_DIR" || exit 1
cargo build --quiet --bin idx
) >"$build_log" 2>&1; then
echo "build failed; see $build_log" >&2
exit 1
fi
fi
if [[ ! -x "$BIN_PATH" ]]; then
echo "idx binary not found or not executable: $BIN_PATH" >&2
exit 1
fi
}
bootstrap_case() {
local label="$1"
shift
local log_file="$LOG_DIR/bootstrap-${label}.log"
if (( DRY_RUN )); then
printf 'bootstrap %-18s %s\n' "$label" "$*"
return 0
fi
if ! (
cd "$ROOT_DIR" || exit 1
env "${BASE_ENV[@]}" "$BIN_PATH" "$@"
) >"$log_file" 2>&1; then
echo "bootstrap failed for $label; see $log_file" >&2
exit 1
fi
}
prepare_environment() {
bootstrap_case "config-init" config init
bootstrap_case "ownership-db" config set ownership.db_path "$OWNERSHIP_DB"
}
discover_ownership_import_url() {
local family="$1"
local log_file="$LOG_DIR/bootstrap-discover-${family}.log"
local url
if (( DRY_RUN )); then
case "$family" in
above1) printf 'https://example.invalid/above1-lamp1.pdf' ;;
above5) printf 'https://example.invalid/above5-lamp1.pdf' ;;
investor-type) printf 'https://example.invalid/investor-type-lamp1.pdf' ;;
*) return 1 ;;
esac
return 0
fi
if ! (
cd "$ROOT_DIR" || exit 1
env "${BASE_ENV[@]}" "$BIN_PATH" -o json ownership discover --family "$family" --limit 1
) >"$log_file" 2>&1; then
echo "bootstrap failed for ownership-discover-$family; see $log_file" >&2
exit 1
fi
url="$(grep -m1 '"pdf_url"' "$log_file" | sed -E 's/.*"pdf_url": "([^"]+)".*/\1/')"
if [[ -z "$url" ]]; then
echo "bootstrap failed for ownership-discover-$family: could not parse pdf_url from $log_file" >&2
exit 1
fi
printf '%s' "$url"
}
prepare_ownership_import_inputs() {
if ! should_run_group "ownership-import"; then
return 0
fi
OWNERSHIP_ABOVE1_URL="$(discover_ownership_import_url above1)"
OWNERSHIP_ABOVE5_URL="$(discover_ownership_import_url above5)"
OWNERSHIP_INVESTOR_TYPE_URL="$(discover_ownership_import_url investor-type)"
}
cache_case_starts_fresh() {
local group="$1"
local label="$2"
[[ "$group" == "cache" && "$label" == *"-warm" ]]
}
reset_cache_for_case() {
local label="$1"
# Cache checks need a clean starting point so the warm/offline/stale trio
# validates TTL-sensitive fallback behavior instead of reusing prior group state.
bootstrap_case "cache-reset-${label}" cache clear
}
sanitize_name() {
printf '%s' "$1" | tr -cs 'A-Za-z0-9._-' '_'
}
command_display() {
local env_spec="$1"
local cmd_spec="$2"
printf 'env %s' "${BASE_ENV[*]}"
if [[ -n "$env_spec" ]]; then
printf ' %s' "$env_spec"
fi
printf ' %s %s' "$BIN_PATH" "$cmd_spec"
}
json_contract() {
if [[ "$2" -ne 0 ]]; then
printf '%s\n' '.error == true and (.code | type == "string") and (.message | type == "string")'
return
fi
case "$1" in
*"stocks quote "*|*"stocks screen"*)
printf '%s\n' 'type == "array" and all(.[]; (.symbol | type == "string") and (.price | type == "number") and has("change_pct"))' ;;
*"stocks history "*)
printf '%s\n' 'type == "array" and all(.[]; (.date | type == "string") and (.close | type == "number") and (.volume | type == "number"))' ;;
*"stocks technical "*)
printf '%s\n' '(.symbol | type == "string") and (.current_price | type == "number") and has("sma200") and (.signals.overall | type == "string")' ;;
*"stocks growth "*)
printf '%s\n' 'has("revenue_growth_pct") and has("earnings_growth_pct") and (.overall_signal | type == "string")' ;;
*"stocks valuation "*)
printf '%s\n' 'has("pe_trailing") and has("roe_pct") and (.overall_signal | type == "string")' ;;
*"stocks risk "*)
printf '%s\n' 'has("debt_to_equity") and has("roa_pct") and (.overall_signal | type == "string")' ;;
*"stocks fundamental "*)
printf '%s\n' '(.symbol | type == "string") and (.growth.overall_signal | type == "string") and (.valuation.overall_signal | type == "string") and (.risk.overall_signal | type == "string")' ;;
*"stocks compare "*)
printf '%s\n' 'type == "array" and all(.[]; (.symbol | type == "string") and (.growth | type == "object") and (.valuation | type == "object") and (.risk | type == "object"))' ;;
*"stocks profile "*)
printf '%s\n' '(.symbol | type == "string") and has("long_name") and has("country") and (.officers | type == "array")' ;;
*"stocks financials "*)
printf '%s\n' '(.instrument.symbol | type == "string") and has("income_statement") and has("balance_sheet") and has("cash_flow")' ;;
*"stocks earnings "*)
printf '%s\n' '(.symbol | type == "string") and (.history | type == "array") and (.forecast | type == "array")' ;;
*"stocks sentiment "*)
printf '%s\n' '(.symbol | type == "string") and (.statistics | type == "array") and all(.statistics[]; (.bullish | type == "number") and (.bearish | type == "number") and (.neutral | type == "number"))' ;;
*"stocks insights "*)
printf '%s\n' '(.symbol | type == "string") and (.highlights | type == "array") and (.risks | type == "array")' ;;
*"stocks news "*)
printf '%s\n' 'type == "array" and all(.[]; (.symbol | type == "string") and (.title | type == "string") and (.url | type == "string"))' ;;
*)
printf '%s\n' 'type == "object" or type == "array"' ;;
esac
}
run_case_line() {
local line="$1"
local group label expected_exit env_spec cmd_spec expect_spec json_expect
local safe_name log_file stdout_file stderr_file output_file status contract baseline_file
local failure=""
local -a env_parts cmd_parts
IFS="$SEP" read -r group label expected_exit env_spec cmd_spec expect_spec json_expect <<< "$line"
if ! should_run_group "$group"; then
return 0
fi
TOTAL_CASES=$((TOTAL_CASES + 1))
safe_name="$(sanitize_name "${group}-${label}")"
log_file="$(printf '%s/%03d_%s' "$LOG_DIR" "$TOTAL_CASES" "$safe_name")"
stdout_file="$log_file.stdout"
stderr_file="$log_file.stderr"
printf '[%02d] %-10s %s\n' "$TOTAL_CASES" "$group" "$label"
if cache_case_starts_fresh "$group" "$label"; then
reset_cache_for_case "$label"
fi
if (( DRY_RUN )); then
printf ' %s\n' "$(command_display "$env_spec" "$cmd_spec")"
return 0
fi
env_parts=()
cmd_parts=()
if [[ -n "$env_spec" ]]; then
read -r -a env_parts <<< "$env_spec"
fi
read -r -a cmd_parts <<< "$cmd_spec"
(
cd "$ROOT_DIR" || exit 1
env "${BASE_ENV[@]}" "${env_parts[@]}" "$BIN_PATH" "${cmd_parts[@]}"
) >"$stdout_file" 2>"$stderr_file"
status=$?
output_file="$stdout_file"
if [[ "$status" -ne 0 ]]; then
output_file="$stderr_file"
fi
if [[ "$status" -ne "$expected_exit" ]]; then
failure="exit $status, expected $expected_exit"
elif [[ "$status" -ne 0 && -s "$stdout_file" ]]; then
failure="error command wrote to stdout"
elif [[ -n "$expect_spec" ]] && ! grep -Fq -- "$expect_spec" "$output_file"; then
failure="missing '$expect_spec' in expected output stream"
elif [[ " $cmd_spec " == *" -o json "* ]]; then
contract="$(json_contract "$cmd_spec" "$status")"
if ! jq -e -s "length == 1 and (.[0] | ($contract) and (${json_expect:-true}))" \
"$output_file" > /dev/null 2>"$log_file.validation"; then
failure="invalid JSON document or output contract"
fi
elif [[ "$label" == "completions-bash" ]] && ! bash -n "$stdout_file"; then
failure="invalid Bash completion script"
fi
if [[ -z "$failure" && "$group" == "cache" ]]; then
baseline_file="$LOG_DIR/cache-${label%%-*}-baseline.json"
if [[ "$label" == *"-warm" ]]; then
jq -S . "$stdout_file" >"$baseline_file"
elif ! jq -S . "$stdout_file" | cmp -s "$baseline_file" -; then
failure="offline/stale data differs from the warmed cache"
elif [[ ! -s "$stderr_file" ]]; then
failure="expired-cache diagnostic missing from stderr"
fi
fi
if [[ -n "$failure" ]]; then
FAILED_CASES=$((FAILED_CASES + 1))
FAILURES+=("$group/$label ($failure) -> $log_file.{stdout,stderr,validation}")
printf ' FAIL %s logs=%s.{stdout,stderr,validation}\n' "$failure" "$log_file"
return 0
fi
PASSED_CASES=$((PASSED_CASES + 1))
printf ' PASS logs=%s.{stdout,stderr}\n' "$log_file"
}
main() {
local case_line
parse_args "$@"
prepare_paths
if (( DRY_RUN == 0 )) && ! command -v jq >/dev/null 2>&1; then
echo "jq is required for smoke output validation; run inside nix develop" >&2
exit 1
fi
printf 'mode: %s\n' "$MODE"
printf 'workdir: %s\n' "$WORKDIR"
printf 'symbol: %s\n' "$SYMBOL"
printf 'compare symbols: %s\n' "$COMPARE_SYMBOLS"
if [[ ${#GROUP_FILTERS[@]} -gt 0 ]]; then
printf 'group filter: %s\n' "${GROUP_FILTERS[*]}"
fi
build_binary
prepare_environment
prepare_ownership_import_inputs
register_cases
if should_run_group "ownership-import"; then
printf 'ownership above1 url: %s\n' "$OWNERSHIP_ABOVE1_URL"
printf 'ownership above5 url: %s\n' "$OWNERSHIP_ABOVE5_URL"
printf 'ownership investor-type url: %s\n' "$OWNERSHIP_INVESTOR_TYPE_URL"
fi
for case_line in "${CASES[@]}"; do
run_case_line "$case_line"
done done
run_case history "stocks history $SYMBOL --period 3mo"
run_case news "stocks news $SYMBOL --limit 5"
run_case screen "stocks screen --filter top-performers --limit 10"
run_case compare "stocks compare $SYMBOL,BBRI"
# BUMI/ADRO/AIMS return non-finite MSN key ratios (Infinity/NaN).
run_case compare-nonfinite "stocks compare BUMI,ADRO,AIMS"
fi
if (( DRY_RUN )); then if selected ownership; then
printf '\ndry-run complete: %d case(s) selected\n' "$TOTAL_CASES" run_case discover "ownership discover --limit 1"
exit 0 if (( ! DRY_RUN )) && [[ -s "$LOG_DIR/$(printf '%02d' "$TOTAL")_discover.stdout" ]]; then
url="$(jq -r '.[0].pdf_url' "$LOG_DIR/$(printf '%02d' "$TOTAL")_discover.stdout")"
run_case import "ownership import --url $url"
run_case releases "ownership releases"
fi fi
fi
printf '\nsummary: %d passed, %d failed, %d total\n' "$PASSED_CASES" "$FAILED_CASES" "$TOTAL_CASES" (( DRY_RUN )) && { echo "dry-run: $TOTAL case(s)"; exit 0; }
printf 'logs: %s\n' "$LOG_DIR" printf '\nsummary: %d passed, %d failed\nlogs: %s\n' "$PASSED" "$((TOTAL - PASSED))" "$LOG_DIR"
if (( ${#FAILURES[@]} > 0 )); then
if (( FAILED_CASES > 0 )); then printf ' - %s\n' "${FAILURES[@]}"
printf 'failures:\n' exit 1
printf ' - %s\n' "${FAILURES[@]}" fi
exit 1
fi
}
main "$@"