The IDX above-1% workbook stores the share code TRUE (TRINITI DINAMIK)
as a boolean cell holding 1, so its holders were imported under ticker
"1". Map t="b" cells to what Excel displays.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 43-case mock smoke duplicated tests/cli.rs, which already runs the
real binary against fixture providers. Replace the flake's smoke-mock
check with a small packaged-cli check (version JSON, bash completions,
one mock quote), and cut scripts/live-smoke.sh from 672 to ~170 lines:
it now only runs shipped commands against live MSN/Yahoo/IDX and
validates output shape. Rewrite docs/SMOKE.md around the Nix gate and
the drift check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Remove FEATURE_SPEC.md, TODO.md, and docs/AGENT_INTEGRATION.md; their
core gaps have shipped. Open work moves to a short ROADMAP.md.
- Remove the CLAUDE.md symlink and rewrite AGENTS.md as a quick guide:
navigation, rules, and the nix flake check gate.
- Merge OWNERSHIP_SELF_HOSTED.md into OWNERSHIP_PUBLISH.md and drop
private host integration notes.
- ARCHITECTURE: add MSN endpoints, clarify Bing ingest is unsupported.
- OWNERSHIP_SYNC: note source.pdf_url also carries XLSX URLs.
- systemd example: --history 2 so snapshots stay under the 10 MiB
client cap.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Describe canonical local and CI verification, cache reuse, output assertions, workflow boundaries, and observed regression results without treating test counts as coverage targets.
Run deterministic installer and publisher checks in Nix, align dev and release dependency artifacts, narrow derivation inputs, and use content-aware CI cache keys. Replace duplicate Cargo gates with a Nix pre-push gate; validate JSON streams and cache data in representative packaged-binary smoke cases.
MSN orders screener lists by 1-year return, but top/worst-performers
re-sort by day change after truncating to --limit. So `--limit 3` sorted
an arbitrary 3 stocks (worst-performers returned -1.33%, +0.25%, +0.95%)
and gave different stocks than `--limit 25`. Fetch the whole list for
these filters, as high-volume and large-cap already do.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ld3fDSDiBzAaQvgZJX5NKM
change, change_pct and volume were `unwrap_or(0)` when the provider
omitted them (MSN quotes and screener rows; Yahoo with no previous
close), so "unknown" read as "unchanged, zero volume". They are now
Option fields that serialize as null when missing. The table shows "-",
and the performers screens sort unknown changes last in both
directions. Present values are unchanged and cached entries still load.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ld3fDSDiBzAaQvgZJX5NKM
Quotes had no timestamp, so a suspended stock's last price looked
current: Yahoo returns WIKA at 204 with regularMarketTime 2025-02-17.
Add `as_of` (RFC 3339 in WIB) from MSN `timeLastTraded` / Yahoo
`regularMarketTime`, also on screener rows. The table prints a note
under any quote whose last trade is over a week old. The field is
additive and defaults to null, so existing cache entries still load.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ld3fDSDiBzAaQvgZJX5NKM
MSN returns suspended names (21 of 896 on 2026-10-02, e.g. WIKA, SUPR)
with symbol, name and market cap but no price. These were reported as
SYMBOLNOTFOUND, which tells an agent the ticker does not exist. Add a
NoMarketData error for this case; unknown tickers stay SYMBOLNOTFOUND.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ld3fDSDiBzAaQvgZJX5NKM
MSN SentimentBrowser returns `[]` for every IDX ticker checked live
(BBCA, BBRI, TLKM), so `stocks sentiment` always failed with
PARSEERROR. Return empty statistics instead; the table prints a
one-line notice and JSON gets `"statistics": []` with exit 0.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ld3fDSDiBzAaQvgZJX5NKM
MSN stamps each IDX trading day at local midnight (17:00Z the previous
day), so every MSN candle was labelled one day early. Yahoo weekly and
monthly bars have the same stamp and were labelled Sunday / last day of
the previous month. Take the date in the exchange offset instead: Yahoo's
meta.gmtoffset when present, else WIB (UTC+7).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ld3fDSDiBzAaQvgZJX5NKM
Move fmt, clippy, tests, cargo package verification and the offline mock
smoke into flake checks built with crane, so the crate dependency build is
its own derivation that is reused until Cargo.lock changes. CI runs a single
nix flake check and restores /nix/store from the GitHub Actions cache via
nix-community/cache-nix-action, replacing the rustup + cargo test job.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013cfWL2GVg97oLfkNvPLaci
--no-default-features failed to compile because bing.rs imports
crate::ownership types.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Q2PufCUdvVZoVW1Dy53js
- --offline now warns when it serves a cache entry past its TTL; before,
arbitrarily old data was returned with no signal at all
- version, cache info and cache clear honour -o json
- ownership resolve map/merge honour -o json
- stocks quote/compare require at least one symbol instead of printing []
- cache schema-mismatch notice goes through runtime::info so --quiet
silences it, instead of a raw debug eprintln
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Q2PufCUdvVZoVW1Dy53js
Addresses review on #33: Yahoo auto-discovers only curl_chrome*; the
ownership fallback also tries Firefox and Safari profiles.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
- Install: add npm/npx (with the pnpm/bun opt-in), platform coverage,
pin example v0.2.4; Cargo builds from source.
- Runtime helpers as a table, corrected: curl-impersonate is needed for
IDX ownership discover/import and Yahoo fundamentals
(IDX_PROVIDER=yahoo), not for stocks history; mutool only for legacy
PDF reports. Everyday use needs neither.
- Quick start and Features now cover ownership (sync, ticker, entity,
search, changes, concentration, cross-holders, graph; XLSX source on
the IDX Data Kepemilikan Saham page) and the full stocks command set
and providers. Every example was run against v0.2.4.
- Documentation index; script test suites.
- docs: example versions -> v0.2.4; npm publish needs 2FA and a fresh
version number.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
Addresses review on #31:
- the snapshot download limit is the manifest's size_bytes (+1, since
ureq fails once a body reaches its limit), capped at 1 GiB; a body
larger than advertised fails during download. New CLI test; the
>10 MiB test also caught the off-by-one.
- a refused (oversized) standalone build removes its sqlite and
manifest from --output-dir.
- docs: downloads are buffered, not streamed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
On 2026-09-25 a snapshot built with --history 5 (15.3 MB) was
published, and `idx ownership sync` failed for every client:
"the response body is larger than request limit: 10485760" (ureq's
default body cap). It was republished by hand with --history 2 (8.2 MB).
- snapshot download: raise the ureq body limit to 1 GiB; the file is
still verified against the manifest's size and SHA-256. The PDF/XLSX
report downloads get a 256 MiB limit for the same reason.
- CLI test: sync a >10 MiB snapshot over HTTP (fails without the fix
with the exact production error).
- build-latest-ownership-snapshot.sh: --max-snapshot-bytes (default
10485760, the limit of released clients <= v0.2.3); an oversized
snapshot fails the build, so the published one stays untouched.
- publish-ownership-snapshot.sh passes --max-snapshot-bytes through;
publisher tests cover the failure and that the release is unchanged.
- docs: size table and guidance.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
Addresses review on #29:
- drop errtrace: a failure inside $(...) printed two RESULT lines
- explicit failures go through fail(), which always prints
RESULT: FAILED stage=...; idx check gets its own `preflight` stage;
the never-failing published-manifest read is no longer a stage
- validate the built latest_as_of_date (`// empty` + date regex) so a
corrupt manifest cannot compare as "newer"
- freshness: validate --now (exit 2) and latest_as_of_date (exit 3)
before calling date
- flake: package an explicit script list instead of scripts/*.sh
- docs: GH_TOKEN prerequisite for the packaged publisher
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
The self-hosted publish timer has not produced an upload since it was
enabled (manifest last updated 2026-07-14, a manual run), and failures
were invisible. Make the job safe to run daily and easy to deploy:
- publish-ownership-snapshot.sh: skip the upload when the published
manifest already has the latest as-of date (pre-check via discover
for XLSX sources, post-build check for legacy PDF); --force to
override. Always ends with one line: RESULT: published|up-to-date
<as-of> or RESULT: FAILED stage=<stage> (exit N). gh release create
no longer requires a git checkout.
- check-ownership-snapshot-freshness.sh: exit 1 when the published
latest_as_of_date is older than --max-age-days (default 40).
- flake: packages.ownership-publisher bundles the scripts with a
pinned idx (curl-impersonate + mupdf) and jq/gh/sqlite/curl as
bin/idx-ownership-publish and bin/idx-ownership-freshness, so hosts
need no checkout, nix develop or cargo build at run time. Built in CI.
- contrib/systemd + docs: daily 06:00 UTC timer, run semantics table,
freshness alarm, packaged usage.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
Addresses review on #27: canonicalize --history ("08" no longer hits
bash octal parsing), run the history jq in a plain command
substitution so a jq failure aborts under set -e, and restore the
schedule-guidance paragraph in OWNERSHIP_SELF_HOSTED.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
- build-latest-ownership-snapshot.sh: discover with a wider limit and
add --history <n> to import the n previous supported above-1% XLSX
months (one per as-of date, oldest first) before the latest, so
`idx ownership changes` works straight from a synced snapshot.
- publish-ownership-snapshot.sh: pass --history through (for the
clan-private systemd timer via extraArgs).
- publish-ownership-snapshot.yml: `history` input (default 5).
- Docs: publish/self-hosted flow now describes the XLSX source,
observed upload days (2nd-3rd of the month), and --history.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
Addresses review on #26:
- strip ?query/#fragment before the .xlsx/.pdf extension check
- skip page entries without a parseable as-of date instead of emitting
an empty publish_date
- warn on stderr when the data page fails or parses to zero entries
while announcement results still exist, instead of silently
degrading to announcement-only results
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
IDX stopped posting the monthly above-1% list as an announcement after
the 2026-05-29 report; June/July/August are only on the Data
Kepemilikan Saham page. `ownership discover` could not see them, and
IDX's Cloudflare now rejects curl_chrome142 on that page and on the
/Media/ XLSX files.
- remote: parse the page's Nuxt payload (Description/Prospectus pairs,
/ escapes), classify above-1% XLSX as supported and above-5% /
investor-type workbooks as unsupported, derive as_of_date from the
Indonesian description. Keep all above-1% entries, only the newest
of the other series. Merge with the announcement results.
- discover JSON gains `format` (xlsx|pdf) and `as_of_date`; page
entries use the as-of date as publish_date (the page shows none).
- import --url accepts .xlsx and downloads via the XLSX validator.
- curl_impersonate: run_owned_with_fallback tries several browser
profiles until the body validates (JSON / page payload / %PDF / PK);
IDX_CURL_IMPERSONATE_BIN still pins one.
- Tests: page fixture parsing, date parsing, payload validation, CLI
discover via a local page server; existing discover tests now point
IDX_OWNERSHIP_DATA_PAGE_URL at a closed port so no test hits IDX.
- Docs: ARCHITECTURE discovery section, AGENTS ownership inputs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
Addresses review on #25: sheet XML omits blank rows, so error messages
now use each <row r=...> number instead of the position; the excerpt
test pins the exact row count so dropped rows fail loudly.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
The February and March 2026 above-1% workbooks name the investor
column INVESTOR_TYPE (codes such as CP, ID); April onward uses
INVESTOR_CLASSIFICATION (words). Treat both as the same field; every
other column stays strict.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
Since June 2026 IDX publishes the monthly "Pemegang Saham di Atas 1%"
list as an XLSX workbook on the Data Kepemilikan Saham page instead of
a PDF announcement. Add a parser for it and accept `.xlsx` in
`ownership import --file`.
- src/ownership/xlsx.rs: zip + quick-xml reader (shared strings incl.
rich text, sheet cells), exact 12-column header match so the daily
above-5% and investor-type/classification workbooks are rejected,
Excel serial dates, numeric shares, percent -> bps with float-noise
rounding. Builds KseiHoldingDraft directly and reuses the entity
normalizers, so investor types stay uppercase like the PDF path.
- Fixtures: trimmed excerpts of the real 2026-08-31 above-1% and
2026-09-23 above-5% workbooks.
- Unit + CLI tests: import, ticker, releases, above-5% rejection.
- Docs: ARCHITECTURE parser pipeline, AGENTS ownership inputs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
- install.sh: POSIX sh installer that detects OS/arch (incl. Rosetta),
resolves the newest vX.Y.Z release via the GitHub API (skipping
ownership-snapshot-current), verifies SHA256SUMS, sanity-runs the
binary, and installs atomically into ~/.local/bin without sudo.
Supports --version/--dir and IDX_* env overrides; prints a PATH
hint, a NixOS flake suggestion, and helper-tool install hints
instead of calling package managers.
- scripts/install-sh-test.sh: fixture tests against a local fake
release server (latest, pinned, piped, checksum mismatch, missing
release, unsupported OS, PATH hint, reinstall).
- ci.yml: install-script job (ShellCheck, sh + dash on Linux, sh on
macOS).
- release.yml: attach install.sh to each release.
- README + docs/INSTALL.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
- release.yml: build static musl Linux binaries (no host glibc
requirement), add darwin-x64, publish SHA256SUMS, allow
workflow_dispatch for an existing tag (v0.2.3 has no binaries yet),
and run build-only on PRs that touch the workflow or installer
- install.js: verify downloads against the release SHA256SUMS
(IDX_BINARY_SHA256 for overrides), add a 60s request timeout, use
stream.pipeline so truncated downloads fail, and skip the download
inside the source checkout
- idx.js: forward SIGTERM/SIGHUP to the binary and keep the signal
exit status
- package.json: drop the prepare script (no Rust toolchain needed to
pack/publish)
- npm-smoke.sh: read the tarball name from npm pack, use grep over rg
- .gitignore: ignore /bin/ and *.tgz
- docs: musl targets, checksums, pnpm/bun opt-in, manual release flow
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
`.agents/setup` now pre-loads the CLI's local data after the build: it
installs the published ownership snapshot (`idx ownership sync`, the
repo's preferred bootstrap path) and warms the file cache for the shipped
`stocks` read commands, so a fresh orb can query ownership data and serve
`--offline` reads immediately.
Setup also adopts the dev shell environment for its remaining steps via
`nix print-dev-env` (the same mechanism the login-shell hook uses), and
the orb guidance block is replaced in place so text updates reach orbs
that already have one without accumulating blank lines.
Amp-Thread-ID: https://ampcode.com/threads/T-01a0ba76-221d-728b-9d68-45ab48750d3d
Co-authored-by: The Diligence Dev <thediligencedev@gmail.com>
Fresh Amp orbs start without a Rust toolchain or Nix, so every agent would
repeat the same install. `.agents/setup` installs Nix (single-user),
realizes the flake dev shell, activates it for the login shells Amp and orb
services start, and warms target/ so the first agent command is incremental.
`.agents/resume` re-checks the toolchain after a pause.
Also ignore .amp/portals/ so orb runtime files stay out of git status.
Amp-Thread-ID: https://ampcode.com/threads/T-01a0ba76-221d-728b-9d68-45ab48750d3d
Co-authored-by: The Diligence Dev <thediligencedev@gmail.com>
