Bump the workspace and npm packages to 0.3.0. The release workflow
builds and attaches idx-mcp binaries next to idx, the npm launcher and
installer pick their binary from package.json so npm/idx-mcp reuses
them, and the npm smoke test covers both packages.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A snapshot republished for the same as-of date (the TRUE ticker fix)
kept the date, latest release, and release count, so sync saw nothing
to do and existing installs kept the bad data without --force.
The manifest gains an optional snapshot.replaces_sqlite_sha256 listing
earlier builds for that date. Queries never write to a synced file, so
a local database byte-identical to a listed build is replaced; one that
matches nothing (local imports or edits) still needs --force. The
publisher fills the list on a same-date rebuild. Older clients ignore
the field.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Phase 4 of docs/DESIGN.md. Eleven grouped tools (quotes, history, screen,
company, analysis, compare, shareholders, investor holdings, search,
ownership changes, status); ownership_sync is listed only with
--allow-sync. Live results keep the Fetched envelope, errors use the CLI's
{error, code, message} body, and investor tools on an empty database
point at the sync path.
idx-core gains Period/Interval FromStr, Investor::status, a ticker
filter on changes, and Display for ErrorCode (now used by the CLI too).
CLI output and goldens are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
comfy-table counted ANSI escapes as cell width, so colored signal and
change cells pushed the right border out of line on a terminal. Its
custom_styling feature measures the visible width instead. Piped output
has no color and is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The IDX above-1% workbook stores the share code TRUE (TRINITI DINAMIK)
as a boolean cell holding 1, so its holders were imported under ticker
"1". Map t="b" cells to what Excel displays.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Phase 3c of docs/DESIGN.md. `ownership` handlers are now argument
parsing, one idx-core call, and rendering.
- investor.rs: Investor (also Idx::investor()) with releases, ticker,
entity, search, cross_holders, concentration, flow, changes, graph,
unresolved, map_alias, merge_entities, discover, sync, and import.
Import orchestration (input resolution, download, SHA-256 dedup,
PDF/XLSX/archive parsing, entity resolution, release write) moves
from the CLI unchanged.
- Typed inputs: ImportSource, HolderSource, ConcentrationRank, and
parse_report_family, with the same error messages.
- CoreConfig gains an ownership_db override; db::open_db_at opens an
explicit path. idx-cli no longer depends on rusqlite, sha2, or ureq
at runtime.
- Behavior note: an invalid `ticker --source` or `concentration --by`
is now rejected before the database file is opened (and created);
the error and exit code are unchanged.
- Fix: building idx-cli without the ownership feature no longer warns
about the ownership-only `blue` style.
All 143 goldens unchanged; 170 core + 7 CLI unit tests and 62 CLI tests
pass; clippy clean for default, release, testing, and no-default
feature sets. Live A/B against main: 16/16 stocks identical; ownership
discover got IDX 403 on both (exit 4 vs 1 is the phase 1 change).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Phase 3b of docs/DESIGN.md. `stocks` handlers are now argument parsing,
one idx-core call, and rendering.
- Idx (client.rs) is the entry point: Idx::new(CoreConfig) with
market(), financial(), and analysis(). Live lookups return
Fetched<T> { data, source, fetched_at, stale } and take a CachePolicy
(Default | Offline | Refresh, from the --offline/--no-cache flags).
- One cached_fetch replaces the CLI's three cache flows; they differed
only in whether a fresh entry is read via get or get_offline when
offline, which yields the same data and diagnostics. Report-building
errors (technical) still skip the stale-cache fallback.
- market.rs: quotes, history (history_source is resolved first so the
CLI's fallback note keeps its position), and a typed screener
(ScreenFilter/ScreenRegion with the same error messages).
- financial.rs: profile, statements, earnings, sentiment, insights,
news, with StatementFilter/EarningsFilter.
- analysis: TechnicalReport moves to analysis::report; Analysis offers
technical, growth, valuation, risk, fundamental, and compare.
- CoreConfig::load() owns provider/exchange/TTL settings from
config.toml and IDX_*; the CLI's IdxConfig adds output and color.
With two different invalid settings at once, the reported one may
differ from before; every single-error message is unchanged.
- Cache gains fresh/any/offline accessors returning fetch metadata.
Unit tests moved with their code (core 157 -> 168, CLI 17 -> 7). All
143 goldens unchanged; live A/B against main: 16/16 stocks identical.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Phase 3a of docs/DESIGN.md. MockProvider moves to api/mock.rs, and it,
the embedded MSN fixture bodies, and the IDX_MOCK_* hooks compile only
with idx-core's new `testing` feature. idx-cli enables it as a
dev-dependency, so `cargo test` keeps the fixture-backed provider while
release binaries no longer embed fixture data or honor
IDX_USE_MOCK_PROVIDER. Parser helpers used only by the mock are gated
the same way.
The packaged-cli flake check no longer relies on the mock; it checks
an offline cache miss (JSON CACHEMISS, exit 3) instead.
Goldens unchanged; clippy is clean with and without `testing`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Phase 2 of docs/DESIGN.md, no behavior change:
- crates/idx-core (library): api, analysis, cache, config file handling
and provider selection, curl_impersonate, error, ownership, runtime,
plus the shared test fixtures. clap derives are behind an optional
`clap` feature so the library does not require clap.
- crates/idx-cli (binary `idx`): cli/, output/, and IdxConfig (output
format and color resolution). Package and binary names are unchanged.
- config.rs is split: file read/write and validation move to core; a
CLI test keeps core's OUTPUT_VALUES in sync with OutputSetting.
- ownership::db::open_db/db_path drop their unused IdxConfig argument.
- One workspace version; shared dependency versions; no third-party
version changes in Cargo.lock. LICENSE is symlinked into each crate.
- flake: workspace version, new source paths, and a per-crate package
check (idx-cli is built from its tarball against packaged idx-core,
since idx-core is not on crates.io yet). CI cache key, release path
filter, and the golden whitespace-hook exclusion follow the new layout.
Parity: all 143 golden outputs match byte for byte without
regeneration; 157 core + 17 CLI unit tests (173 before plus the new
OUTPUT_VALUES sync test), 62 CLI tests pass. Live A/B against main:
16/16 stocks cases identical; ownership discover got IDX 403 on both
(exit 4 vs 1 is the phase 1 exit-code change).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Phase 1 of docs/DESIGN.md:
- Output format resolves -o > IDX_OUTPUT > general.output > auto; auto
is a table on a terminal and JSON when stdout is piped or captured.
`config init` now writes output = "auto".
- JSON is pretty only on a terminal and compact otherwise. New `-o
jsonl` prints one array item per line with the same key order.
- No ANSI color when stdout is not a terminal (direct owo-colors calls
now go through output::style).
- `stocks quote -` and `stocks compare -` read symbols from stdin.
- Exit codes by class: 2 invalid input/config, 3 not found or cache
miss, 4 upstream unavailable, 5 unsupported, 1 other.
Golden changes are limited to those documented kinds and were verified
mechanically: JSON goldens are semantically identical, tables differ
only by removed ANSI codes and the column padding comfy-table had
reserved for them (cell contents identical), exit codes match each
error's class, and help/completions only gain the new --output values.
17 new golden cases cover auto, jsonl, stdin, and the new errors.
tests/cli.rs now pins IDX_OUTPUT=table in its harness (its tests assert
table text and run piped) and parses discover JSON instead of matching
pretty-printed whitespace.
Live A/B against main: 16/16 stocks cases identical; ownership discover
got IDX 403 on both binaries, now exiting 4 instead of 1 as intended.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Phase 0 of docs/DESIGN.md, before any refactor code:
- tests/golden.rs runs every case in tests/golden/cases.toml (126
outputs across help, completions, config, cache, all stocks commands
on MSN and Yahoo mocks, error paths, and ownership queries on a seeded
fixture DB) and compares exit code, stdout, and stderr byte for byte.
Regenerate only with UPDATE_GOLDEN=1 for a reviewed output change.
- scripts/live-smoke.sh --compare <old-idx> runs each live case with
both binaries and fails on differing exit codes, added/missing/retyped
JSON keys, or changed values for slow-moving data.
- Exclude golden files from the whitespace/EOF pre-commit fixers, which
would otherwise rewrite pinned help output.
Both gates were fault-injected: a one-word table change fails exactly
the three history goldens, and a binary dropping profile.long_name fails
only the profile live comparison.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
StatementSection.values was a HashMap, so `stocks financials -o json`
printed line items in a different order on every run. Table output
already sorted them alphabetically; JSON now matches via BTreeMap.
Values are unchanged, and older cache entries deserialize as before.
Found while capturing golden output, which needs deterministic bytes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Accepted design for extracting idx-core as the shared data layer, the
agent-native CLI output contract, idx-mcp tools, packaging, and the
parity gates (golden output tests plus live A/B check) that the
refactor must pass before merging to main.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 43-case mock smoke duplicated tests/cli.rs, which already runs the
real binary against fixture providers. Replace the flake's smoke-mock
check with a small packaged-cli check (version JSON, bash completions,
one mock quote), and cut scripts/live-smoke.sh from 672 to ~170 lines:
it now only runs shipped commands against live MSN/Yahoo/IDX and
validates output shape. Rewrite docs/SMOKE.md around the Nix gate and
the drift check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Remove FEATURE_SPEC.md, TODO.md, and docs/AGENT_INTEGRATION.md; their
core gaps have shipped. Open work moves to a short ROADMAP.md.
- Remove the CLAUDE.md symlink and rewrite AGENTS.md as a quick guide:
navigation, rules, and the nix flake check gate.
- Merge OWNERSHIP_SELF_HOSTED.md into OWNERSHIP_PUBLISH.md and drop
private host integration notes.
- ARCHITECTURE: add MSN endpoints, clarify Bing ingest is unsupported.
- OWNERSHIP_SYNC: note source.pdf_url also carries XLSX URLs.
- systemd example: --history 2 so snapshots stay under the 10 MiB
client cap.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Describe canonical local and CI verification, cache reuse, output assertions, workflow boundaries, and observed regression results without treating test counts as coverage targets.
Run deterministic installer and publisher checks in Nix, align dev and release dependency artifacts, narrow derivation inputs, and use content-aware CI cache keys. Replace duplicate Cargo gates with a Nix pre-push gate; validate JSON streams and cache data in representative packaged-binary smoke cases.
MSN orders screener lists by 1-year return, but top/worst-performers
re-sort by day change after truncating to --limit. So `--limit 3` sorted
an arbitrary 3 stocks (worst-performers returned -1.33%, +0.25%, +0.95%)
and gave different stocks than `--limit 25`. Fetch the whole list for
these filters, as high-volume and large-cap already do.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ld3fDSDiBzAaQvgZJX5NKM
change, change_pct and volume were `unwrap_or(0)` when the provider
omitted them (MSN quotes and screener rows; Yahoo with no previous
close), so "unknown" read as "unchanged, zero volume". They are now
Option fields that serialize as null when missing. The table shows "-",
and the performers screens sort unknown changes last in both
directions. Present values are unchanged and cached entries still load.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ld3fDSDiBzAaQvgZJX5NKM
Quotes had no timestamp, so a suspended stock's last price looked
current: Yahoo returns WIKA at 204 with regularMarketTime 2025-02-17.
Add `as_of` (RFC 3339 in WIB) from MSN `timeLastTraded` / Yahoo
`regularMarketTime`, also on screener rows. The table prints a note
under any quote whose last trade is over a week old. The field is
additive and defaults to null, so existing cache entries still load.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ld3fDSDiBzAaQvgZJX5NKM
MSN returns suspended names (21 of 896 on 2026-10-02, e.g. WIKA, SUPR)
with symbol, name and market cap but no price. These were reported as
SYMBOLNOTFOUND, which tells an agent the ticker does not exist. Add a
NoMarketData error for this case; unknown tickers stay SYMBOLNOTFOUND.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ld3fDSDiBzAaQvgZJX5NKM
MSN SentimentBrowser returns `[]` for every IDX ticker checked live
(BBCA, BBRI, TLKM), so `stocks sentiment` always failed with
PARSEERROR. Return empty statistics instead; the table prints a
one-line notice and JSON gets `"statistics": []` with exit 0.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ld3fDSDiBzAaQvgZJX5NKM
MSN stamps each IDX trading day at local midnight (17:00Z the previous
day), so every MSN candle was labelled one day early. Yahoo weekly and
monthly bars have the same stamp and were labelled Sunday / last day of
the previous month. Take the date in the exchange offset instead: Yahoo's
meta.gmtoffset when present, else WIB (UTC+7).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ld3fDSDiBzAaQvgZJX5NKM
Move fmt, clippy, tests, cargo package verification and the offline mock
smoke into flake checks built with crane, so the crate dependency build is
its own derivation that is reused until Cargo.lock changes. CI runs a single
nix flake check and restores /nix/store from the GitHub Actions cache via
nix-community/cache-nix-action, replacing the rustup + cargo test job.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013cfWL2GVg97oLfkNvPLaci
--no-default-features failed to compile because bing.rs imports
crate::ownership types.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Q2PufCUdvVZoVW1Dy53js
- --offline now warns when it serves a cache entry past its TTL; before,
arbitrarily old data was returned with no signal at all
- version, cache info and cache clear honour -o json
- ownership resolve map/merge honour -o json
- stocks quote/compare require at least one symbol instead of printing []
- cache schema-mismatch notice goes through runtime::info so --quiet
silences it, instead of a raw debug eprintln
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Q2PufCUdvVZoVW1Dy53js
Addresses review on #33: Yahoo auto-discovers only curl_chrome*; the
ownership fallback also tries Firefox and Safari profiles.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
- Install: add npm/npx (with the pnpm/bun opt-in), platform coverage,
pin example v0.2.4; Cargo builds from source.
- Runtime helpers as a table, corrected: curl-impersonate is needed for
IDX ownership discover/import and Yahoo fundamentals
(IDX_PROVIDER=yahoo), not for stocks history; mutool only for legacy
PDF reports. Everyday use needs neither.
- Quick start and Features now cover ownership (sync, ticker, entity,
search, changes, concentration, cross-holders, graph; XLSX source on
the IDX Data Kepemilikan Saham page) and the full stocks command set
and providers. Every example was run against v0.2.4.
- Documentation index; script test suites.
- docs: example versions -> v0.2.4; npm publish needs 2FA and a fresh
version number.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
Addresses review on #31:
- the snapshot download limit is the manifest's size_bytes (+1, since
ureq fails once a body reaches its limit), capped at 1 GiB; a body
larger than advertised fails during download. New CLI test; the
>10 MiB test also caught the off-by-one.
- a refused (oversized) standalone build removes its sqlite and
manifest from --output-dir.
- docs: downloads are buffered, not streamed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
On 2026-09-25 a snapshot built with --history 5 (15.3 MB) was
published, and `idx ownership sync` failed for every client:
"the response body is larger than request limit: 10485760" (ureq's
default body cap). It was republished by hand with --history 2 (8.2 MB).
- snapshot download: raise the ureq body limit to 1 GiB; the file is
still verified against the manifest's size and SHA-256. The PDF/XLSX
report downloads get a 256 MiB limit for the same reason.
- CLI test: sync a >10 MiB snapshot over HTTP (fails without the fix
with the exact production error).
- build-latest-ownership-snapshot.sh: --max-snapshot-bytes (default
10485760, the limit of released clients <= v0.2.3); an oversized
snapshot fails the build, so the published one stays untouched.
- publish-ownership-snapshot.sh passes --max-snapshot-bytes through;
publisher tests cover the failure and that the release is unchanged.
- docs: size table and guidance.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
Addresses review on #29:
- drop errtrace: a failure inside $(...) printed two RESULT lines
- explicit failures go through fail(), which always prints
RESULT: FAILED stage=...; idx check gets its own `preflight` stage;
the never-failing published-manifest read is no longer a stage
- validate the built latest_as_of_date (`// empty` + date regex) so a
corrupt manifest cannot compare as "newer"
- freshness: validate --now (exit 2) and latest_as_of_date (exit 3)
before calling date
- flake: package an explicit script list instead of scripts/*.sh
- docs: GH_TOKEN prerequisite for the packaged publisher
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
The self-hosted publish timer has not produced an upload since it was
enabled (manifest last updated 2026-07-14, a manual run), and failures
were invisible. Make the job safe to run daily and easy to deploy:
- publish-ownership-snapshot.sh: skip the upload when the published
manifest already has the latest as-of date (pre-check via discover
for XLSX sources, post-build check for legacy PDF); --force to
override. Always ends with one line: RESULT: published|up-to-date
<as-of> or RESULT: FAILED stage=<stage> (exit N). gh release create
no longer requires a git checkout.
- check-ownership-snapshot-freshness.sh: exit 1 when the published
latest_as_of_date is older than --max-age-days (default 40).
- flake: packages.ownership-publisher bundles the scripts with a
pinned idx (curl-impersonate + mupdf) and jq/gh/sqlite/curl as
bin/idx-ownership-publish and bin/idx-ownership-freshness, so hosts
need no checkout, nix develop or cargo build at run time. Built in CI.
- contrib/systemd + docs: daily 06:00 UTC timer, run semantics table,
freshness alarm, packaged usage.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
Addresses review on #27: canonicalize --history ("08" no longer hits
bash octal parsing), run the history jq in a plain command
substitution so a jq failure aborts under set -e, and restore the
schedule-guidance paragraph in OWNERSHIP_SELF_HOSTED.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
- build-latest-ownership-snapshot.sh: discover with a wider limit and
add --history <n> to import the n previous supported above-1% XLSX
months (one per as-of date, oldest first) before the latest, so
`idx ownership changes` works straight from a synced snapshot.
- publish-ownership-snapshot.sh: pass --history through (for the
clan-private systemd timer via extraArgs).
- publish-ownership-snapshot.yml: `history` input (default 5).
- Docs: publish/self-hosted flow now describes the XLSX source,
observed upload days (2nd-3rd of the month), and --history.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
Addresses review on #26:
- strip ?query/#fragment before the .xlsx/.pdf extension check
- skip page entries without a parseable as-of date instead of emitting
an empty publish_date
- warn on stderr when the data page fails or parses to zero entries
while announcement results still exist, instead of silently
degrading to announcement-only results
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
