docs: prune stale planning docs and lean out agent guide

- Remove FEATURE_SPEC.md, TODO.md, and docs/AGENT_INTEGRATION.md; their core gaps have shipped. Open work moves to a short ROADMAP.md. - Remove the CLAUDE.md symlink and rewrite AGENTS.md as a quick guide: navigation, rules, and the nix flake check gate. - Merge OWNERSHIP_SELF_HOSTED.md into OWNERSHIP_PUBLISH.md and drop private host integration notes. - ARCHITECTURE: add MSN endpoints, clarify Bing ingest is unsupported. - OWNERSHIP_SYNC: note source.pdf_url also carries XLSX URLs. - systemd example: --history 2 so snapshots stay under the 10 MiB client cap. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 01:35:36 +00:00 · 2026-10-14 07:39:21 +00:00 · 2026-10-14 07:39:21 +00:00 · 1631058b8c
commit 1631058b8c
parent cd410708c0
15 changed files with 206 additions and 1033 deletions
--- a/.codex/skills/idx-release/SKILL.md
+++ b/.codex/skills/idx-release/SKILL.md
@ -16,7 +16,7 @@ ownership snapshot artifacts as a separate release track.
 Read these files before mutating release state:
 - `Cargo.toml` for the package version
- `TODO.md` for the latest smoke findings and release-readiness notes
+- `ROADMAP.md` for known limits that may belong in release notes
 - `README.md` if install or publish notes may need updates
 - `docs/OWNERSHIP_SYNC.md` and `docs/OWNERSHIP_PUBLISH.md` only when checking
  ownership-sync behavior or keeping snapshot and app releases separated
@ -59,13 +59,11 @@ curl -fsSL https://crates.io/api/v1/crates/idx-cli
 For a real release or release-hygiene fix, verify the repo state first.
-Run the standard checks inside `nix develop`:
+Run the canonical gate (fmt, clippy, tests, release build, `cargo package`
 verification, and mock smoke):
 ```bash
-nix develop --command cargo build
+nix flake check -L
 nix develop --command cargo clippy -- -D warnings
 nix develop --command cargo test
 nix develop --command cargo package --locked
 ```
 Run targeted smoke when the release changes user-facing CLI behavior. For
--- a/AGENTS.md
+++ b/AGENTS.md
@ -1,102 +1,42 @@
 # AGENTS.md
-## Project
+`idx-cli` is a Rust CLI (binary `idx`) for Indonesian stock market analysis
-`idx-cli` is a Rust CLI for Indonesian stock market analysis and ownership workflows.
+(`stocks`) and KSEI shareholder ownership (`ownership`).
-The repo currently has two main product areas:
+## Navigation
 - `stocks`: live market data and analysis
 - `ownership`: import/sync once, then query locally from SQLite
 ## Stack
 - Rust stable
 - `clap` 4 for CLI
 - `ureq` 3 for HTTP
 - `comfy-table` and `owo-colors` for output
 - `rusqlite` + bundled SQLite/FTS5 for ownership
 - TOML config in `~/.config/idx/config.toml`
 - file cache in `~/.cache/idx/`
 ## Source Map
 ```text
 src/
-├── main.rs
+├── main.rs, config.rs, cache.rs, error.rs, curl_impersonate.rs
-├── cli/
+├── cli/        clap commands: stocks, ownership, config, cache
-│   ├── stocks.rs
+├── api/        providers: msn/ (default), yahoo/ (history/OHLCV)
-│   ├── ownership.rs
+├── analysis/   technical, fundamental, signals
-│   ├── config.rs
+├── ownership/  xlsx + pdf parsers, archive (ZIP/TXT), remote discovery,
-│   └── cache.rs
+│               snapshot sync, db (SQLite/FTS5), entities, search, graph
-├── api/
+└── output/     table.rs, json.rs
-│   ├── mod.rs
+tests/cli.rs    CLI integration tests; fixtures in tests/fixtures/
-│   ├── types.rs
+scripts/        smoke, installer, and ownership snapshot publisher
 │   ├── yahoo/
 │   └── msn/
 ├── analysis/
 ├── ownership/
 │   ├── archive.rs
 │   ├── db.rs
 │   ├── entities.rs
 │   ├── parser.rs
 │   ├── remote.rs
 │   ├── snapshot.rs
 │   └── types.rs
 ├── output/
 ├── cache.rs
 ├── config.rs
 └── error.rs
 ```
-## Provider Model
+Docs:
- `MSN` is the primary/default provider for quotes, fundamentals, profile, earnings, financials, sentiment, insights, news, and screener data.
+- `docs/ARCHITECTURE.md`: providers, ownership ingest/discovery, error and output contract
- `Yahoo` is the fallback provider for history/OHLCV because MSN history for IDX is still not supported.
+- `docs/CONVENTIONS.md`: types, naming, testing, and git rules
- Current config knobs:
+- `docs/OWNERSHIP_SYNC.md` / `docs/OWNERSHIP_PUBLISH.md`: snapshot consumer contract / maintainer publishing
-  - `IDX_PROVIDER=msn|yahoo`
+- `docs/SMOKE.md`: smoke runner and Nix check coverage
-  - `IDX_HISTORY_PROVIDER=auto|yahoo|msn`
+- `ROADMAP.md`: open work
-## Ownership Model
+## Rules
-Ownership is local-first after bootstrap.
+1. Keep data access provider-driven; no ad hoc fetches in `cli/`.
 2. Keep parse/normalize functions pure.
 3. Tests use fixtures and `IDX_USE_MOCK_PROVIDER=1`; never hit the live network.
 4. Output contract: tables on stdout, one JSON document with `-o json`, errors
   on stderr (JSON-shaped in JSON mode), stdout empty on failure.
 5. Keep ownership queries (`releases`, `ticker`, `changes`, ...) working across
   every ingest path: XLSX, legacy PDF, archive ZIP/TXT, and snapshot sync.
 6. `ownership import --fetch-bing` is intentionally unsupported.
-Preferred bootstrap/update path:
+## Verify
 1. `idx ownership sync`
 2. if no snapshot manifest is available: `idx ownership discover` then `idx ownership import --url <xlsx-or-pdf-url>`
 3. local `--file` imports remain available for manual/fallback use
 Ownership input paths:
 - primary remote source: IDX `above1` holder register — monthly XLSX on the Data Kepemilikan Saham page (since the 2026-05-29 report), legacy PDF announcements before that; `ownership discover` checks both
 - maintained snapshot path: `ownership sync`
 - local file imports: IDX above-1% `.xlsx` (current format since June 2026), legacy `.pdf`, plus local archive `.zip` / `.txt`
 Important scope note:
 - archive ZIP/TXT ingest is a fallback/backstop path, not the primary product ingest surface
 - `ownership import --fetch-bing` is still intentionally unsupported
 ## Working Principles
 1. Keep data access provider-driven where possible; avoid adding new ad hoc fetch paths at the CLI layer.
 2. Prefer pure parse/normalize transforms over hidden state.
 3. Use fixtures in tests; do not hit live network in automated tests.
 4. Preserve the output contract: table to stdout, JSON with `--output json`, errors to stderr.
 5. Treat ownership schema/query compatibility as important: `releases`, `ticker`, and `changes` should keep working across ingest paths.
 ## Verification
 Core verification:
 ```bash
-nix develop
+nix flake check -L                       # canonical gate (fmt, clippy, tests, package, smoke)
-cargo build
+nix develop --command cargo test <name>  # focused loop
-cargo clippy -- -D warnings
+scripts/live-smoke.sh --mode full        # opt-in live providers
 cargo test
 ```
 Smoke tooling:
 ```bash
 scripts/live-smoke.sh
 scripts/live-smoke.sh --mode mock
 scripts/live-smoke.sh --mode full
 ```
 ## Read First
 Start with these repo docs before making changes:
 - `FEATURE_SPEC.md` — active implementation backlog and remaining core gaps
 - `TODO.md` — execution tracker and smoke notes
 - `docs/ARCHITECTURE.md` — provider and ownership flow
 - `docs/OWNERSHIP_SYNC.md` — snapshot sync contract
 - `docs/SMOKE.md` — reusable smoke commands
 - `docs/CONVENTIONS.md` — repo conventions
--- a/CLAUDE.md
+++ b/CLAUDE.md
@ -1 +0,0 @@
 AGENTS.md
--- a/FEATURE_SPEC.md
+++ b/FEATURE_SPEC.md
@ -1,207 +0,0 @@
 # Feature Spec: Current Coverage and Remaining Core Work
 **Status:** Active - updated against the current repo state on 2026-03-31
 **Current focus:** Close the remaining core gaps before expanding into new market and watchlist surfaces
 **Reference:** `origin/dev/rubick` (endpoint reference only, not the current implementation plan)
 ---
 ## Purpose
 This is no longer a "port MSN from zero" spec.
 The CLI already ships MSN-backed support for:
 - `profile`
 - `financials`
 - `earnings`
 - `sentiment`
 - `insights`
 - `news`
 - `screen`
 Use this document to track what is still open in core functionality.
 Use `TODO.md` as the execution log and smoke-history record.
 ---
 ## Verified Current State
 - The canonical automated gate is `nix flake check -L`: unit/module and CLI integration tests, packaged-binary smoke, packaging, lint, and shell regressions. Per-run results belong in `TODO.md`, not a fixed test-count target here.
 - Reusable smoke coverage exists via `scripts/live-smoke.sh`; command groups are documented in `docs/SMOKE.md`.
 - The latest smoke notes in `TODO.md` report passing live table and JSON checks for all shipped `stocks` commands.
 - Cache/offline parity, JSON startup-error handling, screener input validation, and the recent MSN output cleanups have already been completed.
 - KSEI ownership import/query is now verified end-to-end from a local March 2026 PDF file into SQLite, with direct CLI reads through `ownership releases` and `ownership ticker`.
 - Remote ownership ingestion should now be treated as an IDX PDF discovery-and-fetch problem first, not a hardcoded KSEI PDF URL problem: direct IDX announcement PDFs exist, but the hashed asset URL must be discovered from an IDX listing/announcement surface and fetched with browser-like behavior.
 - The CLI now has an explicit `idx ownership discover` surface that enumerates the latest hashed BEI ownership report URLs and defaults to the supported `above1` family.
 - Live verification on `2026-03-29` confirmed that the parser-compatible discovered source is currently the `Pemegang Saham di atas 1% (KSEI)` `lamp1` attachment, and `idx ownership import --url` now works end to end for that discovered BEI PDF.
 - The currently discoverable `above 5%` and `investor-type` BEI families remain different schemas and are now classified and rejected explicitly during import instead of falling through to a generic zero-row parse failure.
 - Ownership smoke coverage now includes a dedicated `ownership-import` group that verifies live supported import plus expected unsupported-family failures.
 - `idx ownership sync` now installs maintained SQLite snapshots via a manifest-driven contract with checksum validation, conservative local replacement rules, and fixture-backed regression coverage for install/no-op/force-refresh behavior.
 - The KSEI archive ZIP/TXT path is now implemented as a local-file maintainer fallback/backstop, with cross-check coverage against the `above1` PDF contract for a shared monthly release.
 This means the main gap is no longer endpoint coverage.
 The remaining work is architecture cleanup, a few correctness edge cases, and selective UX expansion on top of already-shipped commands.
 ---
 ## Coverage Snapshot
 | Area | CLI | Status | Notes |
 | --- | --- | --- | --- |
 | Quotes | `idx stocks quote` | Implemented | Cached, smoke-tested, and covered by integration tests |
 | History | `idx stocks history` | Implemented with provider-specific limits | Yahoo remains the default/auto OHLCV source; explicit MSN history works for supported price-only chart windows |
 | Technical | `idx stocks technical` | Implemented | Uses the cached history path |
 | Growth | `idx stocks growth` | Implemented | Shipped and exercised |
 | Valuation | `idx stocks valuation` | Implemented | Shipped and exercised |
 | Risk | `idx stocks risk` | Implemented | Shipped and exercised |
 | Fundamental | `idx stocks fundamental` | Implemented | Shipped and exercised |
 | Compare | `idx stocks compare` | Implemented | Shipped and exercised |
 | Company profile | `idx stocks profile` | Implemented | Cache/offline parity and fixture-backed coverage are in place |
 | Financial statements | `idx stocks financials` | Implemented | Output cleanup landed, and statement filters now support `--statement income|balance|cashflow` |
 | Earnings | `idx stocks earnings` | Implemented | History/forecast split is rendered, and filters now support `--forecast|--history` with `--annual|--quarterly` |
 | Sentiment | `idx stocks sentiment` | Implemented | Fixture-backed CLI coverage exists |
 | Insights | `idx stocks insights` | Implemented | Summary/highlights/risks/`last_updated` mapping was corrected and tested |
 | News | `idx stocks news` | Implemented | Fixture-backed CLI coverage exists |
 | Screener | `idx stocks screen` | Implemented with gaps | Validation landed; expression/preset workflow is still future work |
 | MSN charts | `idx stocks history --history-provider msn` | Implemented with limits | Supports `--period 1mo|3mo|1y --interval 1d`; MSN provides price-only chart series, so OHLC is synthesized and volume is `0` |
 | KSEI ownership import/query | `idx ownership import --file`, `idx ownership import --url`, `idx ownership releases`, `idx ownership ticker` | Implemented | Local PDF import and SQLite-backed query flow are verified against the March 2026 KSEI release; remote IDX import now works for the discovered `above 1%` `lamp1` BEI attachment, and legacy `above 5%` / `investor-type` BEI report families are rejected explicitly |
 | KSEI archive fallback import | `idx ownership import --file <.zip|.txt>` | Implemented as fallback | Local archive ZIP/TXT ingest maps investor-type/locality buckets into synthetic aggregate holders for validation/backstop use, not the primary ingest surface |
 | Ownership snapshot sync | `idx ownership sync` | Implemented | Manifest-driven SQLite snapshot install with checksum validation, conservative replacement/no-op rules, and publisher helper script |
 ---
 ## Resolved Since The Previous Revision
 The following items should no longer be treated as active backlog in this spec:
 - CLI truth-pass baseline for shipped `stocks` commands
 - Unified cache, offline, stale-cache, and `--no-cache` handling across the core and MSN-only stock commands
 - Rejection of the conflicting `--offline --no-cache` flag combination
 - JSON-aware startup/config failures, not just runtime failures
 - Validation for `stocks screen --filter` and `--region`
 - `profile` output remapping to prefer company/localized fields
 - `insights` output remapping for summary, highlights, risks, and `last_updated`
 - Signed-number and table-label cleanup for `financials`
 - Table-mode cleanup for `earnings`
 - Baseline parser and CLI regression coverage for the shipped MSN-only command set
 - KSEI ownership parser hardening for the March 2026 live PDF layout, including the merged `DATE + SHARE_CODE` segment and `D`/`A` locality markers
 - Real KSEI ownership CLI verification from local file import into SQLite (`7261` rows across `955` tickers on `2026-03-28`)
 - Ownership remote-import hardening for the `above1` contract, including direct-PDF-only `--url` input, discovery status output, explicit legacy-schema rejection, and live ownership-import smoke coverage
 - Ownership snapshot publishing/sync contract, including `idx ownership sync`, manifest/checksum validation, conservative local replacement rules, and the `scripts/build-ownership-snapshot.sh` publisher helper; the publisher now uses an immutable content-addressed SQLite asset, uploads and verifies it before the manifest commit point, and makes history-aware no-op decisions
 - KSEI archive ZIP/TXT fallback ingest for local `.zip`/`.txt` files, including cross-check coverage against the `above1` PDF holder-register fixture and compatible `releases`/`ticker`/`changes` verification
 If any of the above regress, capture that in `TODO.md` as a new finding rather than reopening the old section here wholesale.
 ---
 ## Remaining Core Gaps
 ### P0 - Correctness and architecture
 #### 1. Unify provider and capability flow
 Current state:
 - `src/cli/stocks.rs` still directly constructs `MsnProvider::new(false)` for `profile`, `financials`, `earnings`, `sentiment`, `insights`, `news`, and `screen`.
 - This keeps a split execution path alive even though cache/offline behavior is now mostly unified around `fetch_msn_with_cache`.
 Why it matters:
 - The architecture doc describes provider/capability-based flow, but the CLI still special-cases MSN-only commands at the handler layer.
 - Future features will be harder to extend cleanly if this split remains.
 Done when:
 - Normal command handlers stop constructing `MsnProvider` directly.
 - Provider selection and capability checks are centralized and consistent with `docs/ARCHITECTURE.md`.
 #### 2. Fix screener row hygiene for incomplete data
 Current state:
 - `src/api/msn/map.rs` still defaults missing screener price data to `0.0` when constructing `Quote` rows.
 Why it matters:
 - A zero-priced row is not the same thing as a valid priced stock.
 - This can silently admit incomplete market rows instead of rejecting or filtering them.
 Done when:
 - Screener rows without usable price data are filtered or rejected explicitly.
 - Regression tests cover the chosen behavior.
 #### 3. Decide the fundamentals fallback policy
 Current state:
 - Fundamentals can still fall back to industry-level metrics when company metrics are absent.
 Why it matters:
 - This can produce analysis that looks precise but is actually based on peer or category data.
 Decision needed:
 - Allow the fallback and annotate it clearly, or
 - reject the fallback and surface missing company data explicitly.
 Done when:
 - The policy is explicit in code and reflected in output semantics.
 #### 4. Harden Yahoo reliability edge cases
 Open issues:
 - Yahoo can still return `429` from datacenter IPs intermittently.
 - SMA200 trend output can still show "Insufficient data" when fewer than `200` candles are returned.
 Done when:
 - The retry/fallback story for Yahoo failures is deliberate and documented.
 - SMA200 behavior is either improved or clearly documented as expected.
 ### P1 - UX and output contract cleanup
 Completed on `2026-04-02`:
 - `financials` now supports `--statement income|balance|cashflow`.
 - `earnings` now supports `--forecast|--history` and `--annual|--quarterly`.
 - JSON payload context is less sparse for MSN `earnings`, `insights`, and `news`; each now carries the resolved symbol.
 - `screen` stays under `stocks` for now; revisit a dedicated surface only if `screen query` / `screen presets` grow into a richer workflow.
 Done when:
 - Existing shipped commands are easier to drive without changing product scope.
 ### P2 - Next feature work after the above is green
 Priority order:
 1. Richer financial statements
   - Decide whether to stay with the current single-period model or add multi-period fetch support.
 2. New user-facing surfaces from `TODO.md`
   - `market summary`
   - `market movers`
   - `market sectors`
   - `screen query`
   - `screen presets`
   - `watchlist`
   - `alerts`
 ---
 ## Verification Gate
 Do not treat a core refactor or new provider feature as complete until all of the following are true:
 - `nix flake check -L` passes, including the packaged-binary mock smoke and the relevant behavioral integration scenarios
 - the relevant live smoke groups pass for changed user-facing behavior
 - `TODO.md` is updated with any new smoke finding, regression, or behavior change
 Keep the detailed reusable smoke commands in `docs/SMOKE.md`.
 Do not duplicate per-run results in this spec.
 ---
 ## Endpoint Reference Appendix
 MSN API key (public, embedded in MSN Money website):
 ```
 0QfOX3Vn51YCzitbLaRkTTBadtWpgTN8NZLW0C1SEM
 ```
 Base URLs:
 - `https://assets.msn.com/service/` - core market data (Quotes, Charts, Equities, Earnings, Sentiment, Screener)
 - `https://api.msn.com/msn/v0/pages/finance/` - extended data (key ratios, insights, news feed)
 Keep this appendix for endpoint discovery and future work.
 Use the sections above as the actual implementation plan.
--- a/README.md
+++ b/README.md
@ -145,9 +145,9 @@ Precedence order:
 - [docs/NPM_DISTRIBUTION.md](docs/NPM_DISTRIBUTION.md): npm package and release flow
 - [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md): providers, ownership ingest, and discovery
 - [docs/OWNERSHIP_SYNC.md](docs/OWNERSHIP_SYNC.md): snapshot manifest and sync rules
- [docs/OWNERSHIP_PUBLISH.md](docs/OWNERSHIP_PUBLISH.md) and
+- [docs/OWNERSHIP_PUBLISH.md](docs/OWNERSHIP_PUBLISH.md): maintainer snapshot publishing
  [docs/OWNERSHIP_SELF_HOSTED.md](docs/OWNERSHIP_SELF_HOSTED.md): maintainer snapshot publishing
 - [docs/SMOKE.md](docs/SMOKE.md): smoke tests against live providers
 - [ROADMAP.md](ROADMAP.md): planned work
 ## Development
--- a/ROADMAP.md
+++ b/ROADMAP.md
@ -0,0 +1,33 @@
 # Roadmap
 Open work only. Shipped behavior is described in `README.md` and `docs/`;
 history lives in git.
 ## Stocks
 - `market summary`: IHSG index and market breadth
 - `market movers`: top gainers, losers, and volume
 - `market sectors`: sector performance overview
 - `screen query "<EXPR>"`: filter stocks by expression
 - `screen presets` / `screen run <PRESET>`: built-in screener presets
  (graduate `screen` out of `stocks` only if these grow into a richer workflow)
 - Multi-period financial statements (today `financials` is single-period)
 ## Workflow
 - `watchlist`: named symbol sets usable by other commands
 - `alerts`: price alerts with a daemon
 - CSV/TSV and JSONL output
 - Stdin symbol input for chaining (`... | idx stocks fundamental --stdin`)
 - Additional providers (Alpha Vantage, Twelve Data, IDX official)
 ## Agent integration
 - MCP server exposing the existing commands as tools (planned as `idx-mcp`)
 - JSON Schema export for command output
 - Composite `analyze` command (quote, fundamentals, ownership, sentiment)
 ## Known limits
 - Yahoo can return `429` from datacenter IPs; requests retry with backoff but
  can still fail.
 - `stocks technical` reports the SMA200 trend as unavailable when fewer than
  200 daily candles are returned.
 - MSN history for IDX is price-only (`1mo`, `3mo`, `1y`); Yahoo stays the
  default OHLCV source.
--- a/TODO.md
+++ b/TODO.md
@ -1,257 +0,0 @@
 # idx-cli TODO
 ## ✅ Completed (v0.1.0)
 - [x] Provider abstraction (MarketDataProvider trait, Yahoo impl)
 - [x] `stocks quote <SYMBOL...>` — real-time quotes, multi-symbol
 - [x] `stocks history <SYMBOL>` — historical OHLC data
 - [x] File cache with TTL + offline mode + stale-cache fallback
 - [x] Config system (TOML file, env vars, CLI flags, precedence)
 - [x] Output modes: table (human) + JSON (agent)
 - [x] 52-week range bar in quote table
 - [x] `cache info` / `cache clear`
 - [x] `config init` / `get` / `set` / `path`
 - [x] Pre-commit hooks (fmt, clippy, test)
 - [x] GitHub Actions CI
 - [x] Nix flake + devshell
 - [x] MIT license
 - [x] crates.io metadata
 ## ✅ Completed (v0.1.1)
 - [x] Technical analysis module (SMA, EMA, RSI, MACD, volume ratio)
 - [x] Signal interpretation (bullish/bearish/neutral consensus)
 - [x] `stocks technical <SYMBOL>` — full technical analysis command
 - [x] Colored signal output (green/red/yellow)
 - [x] 1-year lookback for SMA200 coverage
 ## ✅ Completed (v0.1.1 — fundamental suite)
 - [x] `stocks growth <SYMBOL>` — revenue/earnings growth with signals
 - [x] `stocks valuation <SYMBOL>` — PE, PB, ROE, margins, EV/EBITDA with signals
 - [x] `stocks risk <SYMBOL>` — D/E, current ratio, ROA with signals
 - [x] `stocks fundamental <SYMBOL>` — composite growth + valuation + risk
 - [x] `stocks compare <SYM1,SYM2,...>` — side-by-side multi-symbol comparison
 - [x] `analysis/fundamental.rs` — fundamental analysis module (ported from idx-mcp)
 - [x] Yahoo quoteSummary endpoint parser (/v10/finance/quoteSummary)
 - [x] 142 tests passing (94 unit + 48 integration)
 ## ✅ Completed (2026-03-26 — hardening pass)
 - [x] Unified cache, offline, stale-cache, and `--no-cache` behavior across core and MSN-only stock commands
 - [x] Rejected the conflicting `--offline --no-cache` flag combination explicitly
 - [x] Routed startup/config failures through the same JSON-aware error path as runtime failures
 - [x] Validated `stocks screen --filter` and `--region`; invalid values now return errors
 - [x] Added regression tests for startup JSON errors, MSN profile offline/stale-cache behavior, and screener validation
 ## ✅ Completed (2026-03-26 — MSN output cleanup)
 - [x] Re-mapped live MSN `profile` output to prefer localized/company fields for name, description, sector, industry, website, address, and phone
 - [x] Reworked `stocks insights` output to derive a real summary, split highlights vs risks from evaluation status, and populate `last_updated`
 - [x] Fixed signed-number formatting so `stocks financials` table output no longer mangles negative values
 - [x] Added live-like fixtures and regression coverage for `profile`, `insights`, and signed table formatting
 ## 🚧 Next Up
 - [x] Build a reusable live smoke script/checklist for all shipped CLI commands (`scripts/live-smoke.sh`, `docs/SMOKE.md`)
 - [x] Re-run full live smoke for MSN-only commands after the cache/offline, mapping, and formatting fixes (`scripts/live-smoke.sh --mode full --group live-table --group live-json`)
 - [x] Review remaining noisy table output in `financials` and `earnings` beyond the signed-number fix
 - [x] Expand parser/CLI regression coverage for the rest of the MSN-only command set
 - [x] Keep the live-smoke notes below in sync with real command output after each hardening pass
 ## 🚧 Ownership Roadmap Reset (2026-03-29)
 ### Batch 1 — IDX discovery + remote PDF import
 - [x] Verify and document the IDX announcement/listing page that exposes the monthly ownership PDF link
  - Verified official BEI listing page: `https://www.idx.co.id/id/berita/pengumuman/`
  - Verified listing page JSON source used by the site: `GET /primary/NewsAnnouncement/GetAllAnnouncement?keywords=...`
 - [x] Implement discovery/crawler logic for the hashed IDX PDF asset URL instead of hardcoding monthly paths
 - [x] Expose discovery as an explicit `idx ownership discover` CLI surface so the live BEI feed can be inspected without coupling it to import
 - [x] Extract a reusable browser-impersonated fetch helper for ownership downloads by reusing the repo's `curl-impersonate` pattern
 - [x] Wire `ownership import --url` to the impersonated fetch path for IDX-hosted PDFs
 - [x] Keep the current PDF parser/import path as the first production ingest route
 - [x] Add tests and fixtures for IDX announcement-page parsing plus remote PDF fetch failure modes
 - [x] Batch 1 verification: `cargo build`
 - [x] Batch 1 verification: `cargo clippy -- -D warnings`
 - [x] Batch 1 verification: `cargo test`
 - [x] Batch 1 verification: fixture-backed parser/downloader tests cover announcement discovery, hashed URL extraction, and downloader failure cases
 - [x] Batch 1 verification: live `idx ownership discover --family above1 --limit 2` in `nix develop` resolves the current BEI hashed URLs for the parser-compatible `above 1%` report family
 - [x] Batch 1 verification: live end-to-end import succeeds for one discovered BEI ownership PDF URL
 - [x] Batch 1 verification: end-to-end import into a temp ownership DB via `idx ownership import --url ...`, followed by `idx ownership releases` and one ticker query
  - Resolved root cause on `2026-03-29`:
    - the parser was already correct for the holder-register schema; the missing piece was discovery support for the `Pemegang Saham di atas 1% (KSEI)` family
    - the parser-compatible source is the `lamp1` attachment `https://www.idx.co.id/StaticData/NewsAndAnnouncement/ANNOUNCEMENTSTOCK/From_EREP/202603/b9b638e5a8_8928aca255.pdf`
    - the `above 5%` and `investor-type` BEI families remain different schemas; as of `2026-03-30`, they should be treated as legacy / unsupported input rather than new parser targets
  - Post-merge reconfirmation on `2026-03-30`:
    - live `idx ownership discover --family above1 --limit 2` still resolves the `2026-03-10` `above 1%` BEI pair and keeps the `lamp1` attachment as the parser-compatible source
    - live `idx ownership import --url` into an isolated temp DB succeeded again from `b9b638e5a8_8928aca255.pdf`: `7261` rows for `955` tickers, `1` release, `5200` distinct raw investor names, and `5176` canonical entities for `as_of_date=2026-02-27`
    - post-import sanity checks still pass for `idx ownership releases` and `idx ownership ticker AADI --source ksei`
    - parser coverage now includes a compact real `above1` `mutool 1.27.0` `stext` excerpt fixture, not only the synthetic live-like line fixture
 ### Batch 2 — Above-1 hardening and unsupported-input UX
 - [x] Standardize the supported remote-import contract on the `Pemegang Saham di atas 1% (KSEI)` holder-register layout and its `lamp1` attachment shape
 - [x] Add BEI PDF schema classification before parse/import so `ownership import --url` can reject non-holder-register PDFs before the parser runs
 - [x] Improve CLI error messages and fallback behavior for discovery failure, fetch failure, invalid remote content, and known-but-unsupported legacy BEI schema variants
 - [x] Capture live-like fixtures for the current discoverable `investor-type` and `above 5%` BEI attachments (or their `mutool` `stext` extracts) so unsupported-input detection and failure UX are regression-tested
 - [x] Decide and document whether `ownership discover` should default to `above1` output while keeping legacy families available only for diagnostic use
 - [x] Decide and document whether `ownership import --url` accepts only direct PDF URLs or can also accept an IDX listing page as input
 - [x] Add regression coverage for Cloudflare/HTML responses, missing announcement links, duplicate release imports, and unsupported BEI schema detections
 - [x] Batch 2 verification: `cargo build`
 - [x] Batch 2 verification: `cargo clippy -- -D warnings`
 - [x] Batch 2 verification: `cargo test`
 - [x] Batch 2 verification: ownership-focused smoke checks cover successful remote import plus expected failure UX
 ### Batch 3 — Snapshot publishing + sync
 - [x] Design maintained SQLite snapshot publishing after remote IDX import is stable
 - [x] Add `idx ownership sync`
 - [x] Define manifest/checksum/update semantics and local DB replacement rules
 - [x] Add regression coverage for manifest parsing, checksum validation, no-op sync, and forced refresh
 - [x] Batch 3 verification: `cargo build`
 - [x] Batch 3 verification: `cargo clippy -- -D warnings`
 - [x] Batch 3 verification: `cargo test`
 - [x] Batch 3 verification: sync installs into an empty temp data dir, preserves query behavior, and no-ops when already current
 ### Batch 4 — KSEI ZIP/TXT fallback and cross-check path
 - [x] Keep KSEI ZIP/TXT ingest as fallback and validation/backstop work, not the first milestone
 - [x] Define whether the KSEI archive is only a maintainer fallback or a user-facing alternative import source
 - [x] Add cross-check coverage between IDX-PDF-derived output and KSEI-archive-derived output for at least one monthly release
 - [x] Batch 4 verification: `cargo build`
 - [x] Batch 4 verification: `cargo clippy -- -D warnings`
 - [x] Batch 4 verification: `cargo test`
 - [x] Batch 4 verification: fallback ingest produces a compatible SQLite state for `ownership releases`, `ticker`, and `changes`
 ## 🎯 Current Core Work (from FEATURE_SPEC.md)
 ### P0 — Correctness and architecture
 - [x] Unify provider and capability flow so `src/cli/stocks.rs` stops constructing `MsnProvider` directly for MSN-only command handlers
 - [x] Fix screener row hygiene so incomplete MSN screener rows with missing price data are filtered or rejected instead of defaulting to `0.0`
 - [x] Decide and implement the fundamentals fallback policy when company metrics are missing
 - [x] Harden Yahoo reliability edge cases: intermittent `429` handling and documented/intentional SMA200 behavior with fewer than `200` candles
 ### P1 — UX and output contract cleanup
 - [x] Add `stocks financials` filters such as `--statement income|balance|cashflow`
 - [x] Add `stocks earnings` filters such as `--forecast|--history` and `--annual|--quarterly`
 - [x] Review JSON payload consistency where symbol or context fields are still sparse
 - [x] Decide whether `screen` stays under `stocks` long term or graduates into a richer dedicated surface later
 ### P2 — Deferred but real work
 - [x] Add MSN chart/history support through `stocks history --history-provider msn` for supported price-only daily chart windows (`1mo`, `3mo`, `1y`)
 - [ ] Decide whether richer financial statements should stay single-period or grow into multi-period fetch support
 ## 🚀 Publish Readiness (2026-04-03)
 ### P1 — Release blockers
 - [x] Keep the installed binary name as `idx`; `Cargo.toml` already ships `[[bin]] name = "idx"` while the package remains `idx-cli`
 - [x] Expose a default Nix package/app so the documented `nix run github:0xrsydn/idx-cli` path actually works
 - [x] Restrict packaged crate contents so internal repo files like `.claude/`, `CLAUDE.md`, `AGENTS.md`, and agent-planning docs are not shipped to crates.io
 - [x] Document real runtime dependencies and platform assumptions for `cargo install idx-cli`, including `curl-impersonate-chrome` for Yahoo auth and `mutool` for ownership PDF parsing
 - [x] Add release/install verification to CI, at minimum `cargo package` and an install smoke path; include Nix app/package verification if the README keeps advertising `nix run`
 ### P2 — Pre-publish cleanup
 - [x] Wire `--quiet` so non-essential `info:` / `warning:` output is actually suppressed
 - [x] Fail fast on invalid `IDX_CACHE_QUOTE_TTL` / `IDX_CACHE_FUNDAMENTAL_TTL` env values instead of silently ignoring them
 - [x] Propagate `-v` into the history/technical provider path so verbose Yahoo diagnostics can surface
 - [x] Honor XDG overrides consistently for ownership DB/raw download defaults, not only config/cache
 - [x] Tighten `scripts/live-smoke.sh` so it does not silently validate a stale `target/debug/idx` binary
 - [x] Remove or fix the README reference to the non-existent `skills/` directory
 ## 📋 Backlog (per SPEC.md)
 - [ ] `market summary` — IHSG index, market breadth
 - [ ] `market movers` — top gainers/losers/volume
 - [ ] `market sectors` — sector performance overview
 - [ ] `screen query "<EXPR>"` — filter stocks by expression
 - [ ] `screen presets` / `screen run <PRESET>` — built-in screener presets
 - [ ] `watchlist` commands — create, manage, live watch
 - [ ] `alerts` system (v0.2+) — price alerts with daemon
 - [x] `completions <SHELL>` — shell completion generation
 - [ ] CSV/TSV output formats
 - [ ] Additional providers (Alpha Vantage, Twelve Data, IDX official)
 ## 🔬 Latest Smoke Findings (2026-04-02)
 - [x] Behavioral coverage consolidation on `2026-10-02`: reduced the mock smoke matrix from 62 launch-heavy cases to 43 checks with separate stdout/stderr, single-document JSON validation, fixture values, and exact cache-data comparisons. Consolidated CLI tests from 89 to 62 stronger scenarios; retained all 173 unit/module regressions.
 - [x] Strengthened CLI workflows cover cache misses/expiry/provider isolation, filtered and non-finite financial data, XLSX/ZIP ownership imports and queries, snapshot install/no-op/force, rejected updates preserving database bytes and query results, and 503-candidate screener ranking/full offline caches across four filters.
 - [x] The stricter JSON checks exposed and fixed stderr pollution in unsupported `ownership import --fetch-bing` and totally failed `stocks compare`; partial comparisons still return available data and warnings, with `--quiet` suppressing those diagnostics.
 - [x] Verification: all eleven checks passed via `nix flake check -L --keep-going` on x86_64-linux (173 unit/module tests, 62 CLI tests, 43 packaged-binary smoke cases). Throwaway fault injection proved malformed JSON, extra documents, wrong-stream output, and changed cache data are rejected. Direct packaged-CLI runs confirmed the 503-candidate/offline-prefix boundary, incomplete-total rejection without cache population, and both clean JSON error fixes. Temporary verification fixtures and injectors were removed.
 - [x] Nix verification consolidation on `2026-10-02`: all eleven flake checks pass on x86_64-linux, including publisher regressions, installer `sh`/`dash` scenarios, ShellCheck, 173 unit/module tests, 89 CLI tests, and 62 mock smoke cases. CI and the pre-push hook use `nix flake check -L`; native macOS installer coverage is retained.
 - [x] Checks and crate verification now share dev-profile dependency artifacts, while the installed application retains a separate release cache. Package verification logs confirm only `idx-cli` recompiles instead of rebuilding third-party dependencies. Narrowed source inputs and content-aware CI cache keys avoid unrelated rebuilds and allow source revisions to save new cached outputs.
 - [x] Cache verification on `2026-10-02`: README/docs edits invalidated only the package check; the other ten derivations stayed unchanged. The real Prek pre-push gate passed, and an unchanged `nix flake check -L` completed in 11.69 seconds with no builds or compilation.
 - [x] Combined PR verification on `2026-10-02`: JSON/offline-cache fixes, Nix/Crane CI, and market-data fixes pass `nix flake check -L` on x86_64-linux (173 unit tests, 89 CLI tests, 62 mock smoke cases), plus the publisher and `sh` installer regression scripts.
 - [x] Fixed the integration review's screener truncation: full-list queries now re-request MSN's reported total rather than cap candidates at 500, reject incomplete totals, and use separate `all` cache keys. Live US top-performers returned 600 unique stocks; offline limits 3 and 25 matched the full result's prefixes. One upstream HTTP 499 occurred before the successful live check.
 - [x] Removed the committed `result-1` Nix output symlink and ignored numbered result links.
 - [x] Publish hardening batch on `2026-09-25`: `scripts/publish-ownership-snapshot.sh` now uploads an immutable content-addressed SQLite asset (date + sha256) and verifies it before publishing the manifest as the final commit point, so a partial upload no longer leaves a manifest that points at a missing asset; the manifest is trusted for a no-op only when the referenced asset matches the manifest SHA-256 and size at the configured repo/tag, and GitHub auth/network errors never become "up-to-date"
 - [x] Publish integrity follow-up on `2026-09-25`: no-op and upload verification now check the full SHA-256, not just the filename and size; the publisher uses GitHub's `sha256:` asset digest when present and falls back to `gh release download` + local hashing for legacy assets, rejects a `download_url` that names a different repo/tag, and refuses to publish the manifest when the uploaded asset's SHA-256 does not match
 - [x] `--history` is bounded (maximum `1000`, leading zeros stripped before conversion) so an enormous value cannot wrap negative and look already satisfied
 - [x] The GitHub publish workflow now builds the packaged `idx-ownership-publisher` and routes publication through it with inputs passed as environment variables; the workflow artifact upload uses `if-no-files-found: ignore` so a legitimate up-to-date run does not fail
 - [x] `--keep-workdir` no longer prints after the `RESULT:` line: the retained path is announced when the workdir is created and the EXIT cleanup is silent
 - [x] `--history` changes are now honored for same-date data: the publisher compares `1 + min(--history, available XLSX months)` against the published `release_count` before and after building, validates `--history` before early exits, and does not rebuild on every run when the source has fewer historical months than requested
 - [x] New offline regression harness `scripts/publish-ownership-snapshot-test.sh` (wired into `.github/workflows/ci.yml`) covers successful publish/no-op, partial SQLite upload and retry, manifest upload failure and retry, forced same-date refresh preserving the previous artifact, a missing referenced asset forcing republish, same-size remote corruption with and without digest metadata, upload corruption rejected before the manifest commit (digest and digest-less), a manifest URL for a different repo/tag, legacy PDF post-build no-op, history upgrade, limited available history, `--history` bounds, `--keep-workdir` output ordering, and auth/network failure not becoming up-to-date
 - [x] Verification on `2026-09-25`: `bash -n` on the changed shell scripts and `scripts/publish-ownership-snapshot-test.sh` all passed
 - [x] `ownership sync` now falls back to the built-in published manifest URL on `2026-04-12`, so plain `idx ownership sync` works on a clean machine once the public snapshot release is available
 - [x] Ownership snapshot publishing now has a live-source maintainer path on `2026-04-11`: `scripts/build-latest-ownership-snapshot.sh` discovers the current supported `above1` IDX/KSEI PDF, imports it into an isolated temp DB, emits release-ready SQLite + manifest artifacts, and records the source IDX metadata in the manifest
 - [x] Verification on `2026-04-11`: `nix develop --command cargo test`, `bash -n scripts/build-latest-ownership-snapshot.sh`, and a real `nix develop --command scripts/build-latest-ownership-snapshot.sh --idx-bin ./target/debug/idx --output-dir tmp/ownership-snapshot-live` run all passed; the current live source produced `7214` rows for `955` tickers with `as_of_date=2026-03-31`
 - [x] Final release-hygiene pass on `2026-04-06`: crate metadata now declares `rust-version = 1.85`, README install docs now spell out Cargo helper-runtime expectations plus persistent `nix profile install`, and CI install smoke now runs the mock smoke matrix against the installed binary instead of only checking `idx version`
 - [x] Verification on `2026-04-06`: `nix develop --command cargo build`, `nix develop --command cargo clippy -- -D warnings`, `nix develop --command cargo test`, `nix develop --command cargo package --allow-dirty --locked`, and `scripts/live-smoke.sh --bin ./tmp/release-install/bin/idx --no-build --mode mock` all passed
 - [x] Publish review blocker batch on `2026-04-05`: the default Nix app/package path builds again without relying on an untracked runtime module, `ownership import --force` now re-imports the same release SHA atomically, and current ownership views (`ticker`, `entity`, `cross-holders`, `concentration`, `graph`) now scope KSEI data to the latest imported release instead of blending historical snapshots
 - [x] Verification on `2026-04-05`: `nix develop --command cargo build`, `nix develop --command cargo clippy -- -D warnings`, `nix develop --command cargo test`, `nix build .#default`, `nix develop --command cargo package --allow-dirty --locked`, fresh `cargo install --path . --locked --root tmp/release-install`, `./tmp/release-install/bin/idx version`, and `nix run .#default -- version` all passed
 - [x] Publish P2 cleanup batch on `2026-04-03`: `--quiet` now suppresses non-essential CLI warnings/info, invalid cache TTL env vars fail during startup, `-v` reaches Yahoo history diagnostics, ownership default DB/raw paths honor XDG overrides, and `scripts/live-smoke.sh` now refuses a stale `target/debug/idx` when build refresh is skipped
 - [x] Verification on `2026-04-03`: `nix develop --command cargo build`, `nix develop --command cargo clippy -- -D warnings`, `nix develop --command cargo test`, `bash -n scripts/live-smoke.sh`, and `scripts/live-smoke.sh --mode mock --dry-run` all passed
 - [x] Publish blocker batch on `2026-04-03`: crate packaging now excludes repo-internal agent harness files, the flake exports a default package/app for `nix run`, README install docs now describe runtime helper dependencies, and CI now checks package/install surfaces
 - [x] Verification on `2026-04-03`: `nix develop --command cargo build`, `nix develop --command cargo clippy -- -D warnings`, `nix develop --command cargo test`, `nix develop --command cargo package --allow-dirty --locked`, fresh `cargo install --path . --locked --root tmp/release-install`, `./tmp/release-install/bin/idx version`, `nix build .#default`, and `nix run .#default -- version` all passed
 - [x] P0 provider/capability routing is now centralized through `SelectedProvider`; `stocks` handlers no longer construct `MsnProvider` directly for MSN-only commands
 - [x] MSN fundamentals now reject industry-only metrics with an explicit unsupported error; the normal MSN mock/live path uses company metrics only
 - [x] MSN screener parsing now drops rows with missing/invalid/non-positive price data instead of synthesizing `0.0` price rows
 - [x] Yahoo `429` retry/backoff is now centralized and regression-tested for chart and quote-summary fetches
 - [x] `stocks technical` table output now says `Trend unavailable (need at least 200 daily candles)` when fewer than `200` daily candles are available
 - [x] Verification on `2026-04-02`: `nix develop --command cargo build`, `nix develop --command cargo clippy -- -D warnings`, `nix develop --command cargo test`, `scripts/live-smoke.sh --mode mock`, and `scripts/live-smoke.sh --group live-table --group live-json --group routing --group cache --group errors` all passed (`tmp/live-smoke/20260402-122215`, `tmp/live-smoke/20260402-122218`)
 - [x] Live smoke passed for shipped `stocks` commands: `quote`, `history`, `technical`, `growth`, `valuation`, `risk`, `fundamental`, `compare`, `profile`, `financials`, `earnings`, `sentiment`, `insights`, `news`, `screen`
 - [x] Yahoo routing verified for live `quote` and `history`
 - [x] `stocks history --history-provider msn` now works for supported MSN price-only chart windows; Yahoo remains the auto history source for full OHLCV
 - [x] `ownership releases` works with a writable `ownership.db_path` and an empty DB
 - [x] Regression coverage now verifies offline/cache parity for MSN-only commands (`stocks profile BBCA`)
 - [x] `--offline --no-cache` now fails fast as an invalid flag combination instead of serving cache
 - [x] Config/startup failures now honor the JSON error contract (`IDX_PROVIDER=bogus idx -o json version`)
 - [x] Invalid `stocks screen --filter/--region` values now return validation errors
 - [x] Added reusable smoke runner/checklist for shipped CLI surfaces (`scripts/live-smoke.sh`, `docs/SMOKE.md`)
 - [x] Reusable smoke runner passes on deterministic mock suites: `general`/`cache`/`routing`/`errors`/`ownership` = 32/32 and shipped `stocks` mock matrix = 30/30 (`tmp/live-smoke/20260326-194907`, `tmp/live-smoke/20260326-194914`)
 - [x] Full live MSN-only smoke now passes via runner for both table and JSON surfaces: 30/30 (`tmp/live-smoke/20260326-195853`)
 - [x] Targeted no-cache live checks confirm `profile`, `insights`, and `financials` fixes against real MSN responses
 - [x] Live `stocks profile BBCA` no-cache output now populates company/localized fields instead of the sparse top-level fallback
 - [x] Live `stocks insights BBCA` JSON now returns a mixed-signal summary plus non-empty `last_updated`
 - [x] Live `stocks financials BBCA` table no longer renders malformed negative numbers
 - [x] Re-run full live smoke to reconfirm all MSN-only commands after the latest hardening fixes
 - [x] `stocks financials BBCA` table now trims ISO timestamps from section headers and humanizes raw line-item keys
 - [x] `stocks earnings BBCA` table now splits history vs forecast and formats annual periods, revenue values, and dates for table mode
 - [x] `stocks financials` now supports section filters via `--statement income|balance|cashflow`, with filtered table output and JSON sections rendered as `null` when intentionally excluded
 - [x] `stocks earnings` now supports `--forecast|--history` and `--annual|--quarterly`, so table and JSON output can be scoped without changing the cached source payload
 - [x] JSON payload context is now less sparse for MSN `earnings`, `insights`, and `news`; each payload now carries the resolved stock symbol even when sourced from older cache entries
 - [x] `stocks financials` JSON now normalizes `instrument.symbol` to the exchange-qualified ticker (for example `BBCA.JK`), and older cached payloads are backfilled at read time so users do not need to clear cache after upgrading
 - [x] Product decision on `2026-04-02`: keep `screen` under `stocks` for now; revisit a dedicated surface only when `screen query` / `screen presets` graduate from backlog into a richer workflow
 - [x] Fixture-backed parser and CLI JSON regression coverage now covers the remaining MSN-only `sentiment`, `news`, and `screen` commands
 - [x] Fresh post-coverage live MSN smoke rerun still passes for table and JSON surfaces: 30/30 (`tmp/live-smoke/20260327-163403`)
 - [x] MSN key-ratios parsing now tolerates stringified non-finite numeric sentinels (`Infinity`, `-Infinity`, `NaN`) as missing data instead of failing the entire fundamentals payload
 - [x] Live direct CLI repros on `2026-04-13` confirmed the original failure class on `BUMI`, `ADRO`, and `AIMS`; an opt-in `scripts/live-smoke.sh --group live-nonfinite` group now covers those real ticker paths
 - [x] Added `scripts/audit-msn-fundamentals.sh` for a full CLI valuation sweep across the IDX MSN symbol map; keep it as a provider-health audit, not part of the default smoke baseline
 - [x] Real KSEI PDF import from local file now works again: `ownership import --file /Users/rasyidanakbar/Downloads/ksei_raw_data.pdf` imported `7261` rows for `955` tickers on `2026-03-28`, replacing the previous `6`-row/`1`-ticker failure mode
 - [x] KSEI parser no longer depends on the old hardcoded column bounds fixture layout; it now reconstructs rows from `mutool` line output and handles the live `DATE + SHARE_CODE` merged segment plus `D`/`A` locality markers
 - [x] A real IDX-hosted March 2026 ownership PDF URL was verified on `2026-03-29`, but only through `curl-impersonate` inside `nix develop`; plain `curl` still returns `403` from Cloudflare for the same asset
 - [x] The current repo now has a concrete IDX-first roadmap: discover the hashed PDF URL from IDX pages, fetch it with browser impersonation, and only then layer snapshot publishing and `ownership sync`
 - [x] The KSEI archive (`https://web.ksei.co.id/archive_download/holding_composition`) was verified as a secondary upstream that exposes monthly ZIP files and remains useful for fallback/cross-check work
 - [x] Remote IDX discovery now targets the official BEI `Pengumuman` feed (`/id/berita/pengumuman/`) and its backing JSON endpoint (`/primary/NewsAnnouncement/GetAllAnnouncement`), which exposes hashed attachment URLs for ownership-related reports
 - [x] New `ownership discover` CLI output now shows the latest hashed BEI ownership URLs across the current `above 5%`, `above 1%`, and `investor-type` report families
 - [x] `ownership import --url https://www.idx.co.id/...pdf` now uses the same browser-impersonated download path as Yahoo auth
 - [x] Reverse-engineering the current BEI feed shows the discoverable `Pemegang Saham di atas 1% (KSEI)` `lamp1` attachment matches the known-good raw KSEI holder-register layout and now imports successfully
 - [x] Live `ownership import --url` against the discovered `above 1%` BEI `lamp1` attachment (`b9b638e5a8_8928aca255.pdf`) now succeeds end to end: `7261` rows for `955` tickers on `2026-03-29`, followed by successful `ownership releases` and `ownership ticker AADI --source ksei`
 - [x] Product scope decision on `2026-03-30`: standardize supported remote import on the discoverable `above 1%` holder-register family; treat `above 5%` and `investor-type` PDFs as legacy / unsupported input
 - [x] Live `ownership import --url` checks against the currently discoverable `above 5%` and `investor-type` BEI PDFs still fail with `no KSEI rows parsed from PDF`, which is expected until unsupported-family detection / rejection UX lands
 - [x] Live `mutool` inspection of the current discoverable `investor-type` BEI attachment shows a stock-level aggregate matrix (`DATE`, `STOCK_CODE`, `NUMBER_OF_SHARES`, investor-type columns, holder-size buckets), not the holder-level KSEI register schema the current parser imports
 - [x] Live `mutool` inspection of the current discoverable `above 5%` BEI attachment shows a member/tampungan report (`INVS`, member names, `KSEI UNTUK CLOSED MEMBER-...` labels), not the raw KSEI holder-register layout
 - [x] `ownership discover` now defaults to the supported `above1` family, surfaces per-URL importability status, and orders the importable `lamp1` attachment first so `--limit 1` yields the current supported PDF URL
 - [x] `ownership import --url` now accepts only direct PDF URLs; IDX listing/announcement page URLs fail fast with guidance to run `ownership discover`
 - [x] Valid-but-unsupported BEI PDFs now fail before row parsing with explicit schema-aware errors (`announcement_wrapper`, legacy `above5`, legacy `investor-type`) instead of the old generic `no KSEI rows parsed from PDF` path
 - [x] Regression coverage now covers default `ownership discover` behavior, status visibility, listing-page rejection, duplicate SHA imports, and explicit unsupported-schema detection with compact `stext` fixtures plus fake-`mutool` CLI tests
 - [x] New `ownership-import` smoke coverage now discovers the current live `above1`/`above5`/`investor-type` URLs, imports the supported `above1` attachment successfully, and confirms the legacy families fail with explicit unsupported-schema UX (`tmp/live-smoke/20260330-160201`)
 - [x] New ownership snapshot sync coverage now verifies manifest parsing, checksum validation, install into an empty temp data dir, preserved query behavior for `releases`/`ticker`/`changes`, no-op sync when current, and `--force` refresh via fixture-backed local manifests on `2026-03-31`
 - [x] KSEI archive ZIP/TXT fallback import now works through `ownership import --file` for local `.zip` and `.txt` inputs, using synthetic investor-type/locality aggregate holders as a maintainer backstop rather than the primary product ingest
 - [x] Batch 4 coverage now cross-checks the live-like `2026-02-27` `above1` PDF fixture against the matching KSEI archive bucket excerpt and verifies fallback import/query behavior for `releases`, `ticker`, and `changes`
 ## 🐛 Known Issues
 - [ ] Yahoo Finance returns 429 from datacenter IPs occasionally
 - [ ] SMA200 trend shows "Insufficient data" if Yahoo returns < 200 candles
 - [x] KSEI ownership parser no longer leaks adjacent columns into `INVESTOR_NAME` on the March 2026 live PDF; direct CLI verification now returns clean holders like `AGUNG PERKASA INVESTINDO` / `CP` / `L`
 - [x] Yahoo quoteSummary crumb auth — fixed via curl-impersonate-chrome (curl_chrome131)
  - fc.yahoo.com → A3 cookie + query1 getcrumb → crumb, both sent to quoteSummary
  - Requires nixpkgs#curl-impersonate-chrome in PATH (added to flake.nix + clan-private)
--- a/contrib/systemd/idx-ownership-snapshot-publish.service
+++ b/contrib/systemd/idx-ownership-snapshot-publish.service
@ -9,4 +9,4 @@ Environment=HOME=/var/lib/idx-ownership-snapshot
 EnvironmentFile=/etc/idx-ownership-snapshot.env
 # Build once with `nix build github:0xrsydn/idx-cli#ownership-publisher -o /opt/idx-ownership-publisher`.
 # Idempotent: exits 0 with "RESULT: up-to-date" when nothing newer exists.
-ExecStart=/opt/idx-ownership-publisher/bin/idx-ownership-publish --output-dir /var/lib/idx-ownership-snapshot/current --repo 0xrsydn/idx-cli --release-tag ownership-snapshot-current --history 5
+ExecStart=/opt/idx-ownership-publisher/bin/idx-ownership-publish --output-dir /var/lib/idx-ownership-snapshot/current --repo 0xrsydn/idx-cli --release-tag ownership-snapshot-current --history 2
--- a/docs/AGENT_INTEGRATION.md
+++ b/docs/AGENT_INTEGRATION.md
@ -1,149 +0,0 @@
 # Agent Integration Roadmap
 **Status:** Planning — prerequisites are the remaining P0/P1 core gaps in `FEATURE_SPEC.md`
 **Goal:** Make `idx-cli` a first-class tool for AI agent workflows (Claude Code, OpenClaw, MCP clients)
 ---
 ## Current Foundation
 The CLI already has the building blocks:
 - Structured JSON output (`-o json`) on every command
 - Trait-based provider abstraction (mockable, extensible)
 - Rich typed domain models (`Quote`, `Fundamentals`, `CompanyProfile`, `OwnershipHolder`, etc.)
 - Structured error codes in JSON mode
 - File-based cache with TTL, offline, and stale-cache fallback
 - Feature-gated modules (ownership behind `--features ownership`)
 ---
 ## Tier 1 — MCP Server Mode (highest leverage)
 Add `idx mcp-serve` to expose existing commands as MCP tools over stdio.
 - Use `rust-mcp-sdk` crate for the stdio JSON-RPC transport
 - Each existing command maps to one MCP tool (e.g. `get_stock_quote`, `get_financials`, `screen_stocks`, `get_ownership_ticker`)
 - Existing JSON serialization becomes the tool response body — minimal new code
 - Tool input schemas derive from clap structs or parallel types
 - Configure in Claude Code:
  ```json
  { "mcpServers": { "idx": { "command": "idx", "args": ["mcp-serve"] } } }
  ```
 **Why this first:** One feature makes `idx-cli` instantly usable from Claude Code, Claude Desktop, OpenClaw, and any MCP-compatible client. No wrapper scripts needed.
 **Reference:** `financial-datasets/mcp-server` (1.7k stars), `tradingview-mcp` (500 stars), `twsemcp` (Taiwan Stock Exchange MCP — closest regional analog), `rust-mcp-sdk` (Rust MCP crate).
 ---
 ## Tier 2 — Composability
 ### Stdin symbol input
 ```bash
 idx stocks screen --filter pe_lt:15 -o json | jq -r '.[].symbol' | idx stocks fundamental --stdin -o json
 ```
 Agents generate symbol lists dynamically. Piping enables chained workflows without temp files.
 ### JSONL output
 ```bash
 idx -o jsonl stocks quote BBCA BBRI
 ```
 Newline-delimited JSON — better for streaming and piping than pretty-printed arrays.
 ### Batch mode
 ```bash
 idx batch commands.jsonl
 ```
 Where each line is a command spec:
 ```json
 {"command": "stocks/quote", "symbols": ["BBCA", "BBRI"]}
 {"command": "ownership/ticker", "symbol": "BBCA", "source": "ksei"}
 ```
 Reduces process spawn overhead when agents run many queries in sequence.
 ---
 ## Tier 3 — Schema Export
 ```bash
 idx schema stocks/quote
 ```
 Dumps the JSON Schema for a command's output. Agents use this to understand available fields without trial-and-error. Document which fields are guaranteed present vs. optional.
 ---
 ## Tier 4 — Agent-Oriented Domain Features
 ### Watchlist as named symbol sets
 ```bash
 idx watchlist create banks BBCA,BBRI,BMRI,BNGA
 idx stocks quote --watchlist banks
 ```
 Agents build and reuse curated lists across sessions.
 ### Composite analysis
 ```bash
 idx stocks analyze BBCA -o json
 ```
 Returns quote + fundamentals + ownership + sentiment in one call. Saves agents 4+ tool calls and reduces token overhead.
 ### Diff / change detection
 ```bash
 idx stocks quote BBCA --diff
 ```
 Compares against cached previous values and surfaces deltas. Agents monitor without maintaining their own state.
 ### CSV/TSV export
 ```bash
 idx -o csv stocks screen --filter pe_lt:15
 ```
 For agent pipelines that need tabular data (pandas, spreadsheets, database import).
 ---
 ## Tier 5 — Integration Hooks
 - **Webhook primitives** — `idx watch BBCA --threshold price_gt:5000 --webhook http://...` for event-driven agent architectures
 - **Session context** — `idx context set research-session-1` to tag cache entries, enabling isolated research sessions that don't pollute each other
 ---
 ## Execution Order
 | Priority | Feature | Effort | Impact |
 | --- | --- | --- | --- |
 | 1 | MCP server mode | Medium | Unlocks all MCP clients |
 | 2 | Stdin piping + JSONL | Small | Composability for chaining |
 | 3 | Schema export | Small | Agent self-discovery |
 | 4 | Watchlist + composite | Medium | Reduces agent round-trips |
 | 5 | CSV export | Small | Tabular pipeline support |
 | 6 | Webhooks + sessions | Large | Event-driven architecture |
 ---
 ## Prerequisites
 Close the remaining core gaps first (`FEATURE_SPEC.md` P0/P1):
 1. Unify provider and capability flow
 2. Fix screener row hygiene
 3. Decide fundamentals fallback policy
 4. Harden Yahoo reliability edge cases
 5. Add `financials` and `earnings` filter flags
--- a/docs/ARCHITECTURE.md
+++ b/docs/ARCHITECTURE.md
@ -34,6 +34,13 @@
 - **MSN Finance** — default provider. Rich data: quotes, fundamentals, profile, earnings, financials, sentiment, insights, news, screener, and explicit price-only chart history for supported IDX windows.
 - **Yahoo Finance** — default/auto history source. Reliable OHLCV data via `/v8/finance/chart/`.
 ### MSN Endpoints
 - `https://assets.msn.com/service/`: quotes, charts, equities, earnings,
  sentiment, screener
 - `https://api.msn.com/msn/v0/pages/finance/`: key ratios, insights, news feed
 The public API key embedded in MSN Money lives in `src/api/msn/client.rs`.
 ### Hybrid History Strategy
 When `history_provider = auto` (default):
 1. Use Yahoo for history because it provides full OHLCV candles.
@ -73,7 +80,7 @@ Factory functions:
 Unlike the `stocks` module (live-fetch), ownership is **import-then-query**:
-1. `idx ownership import` — ETL pipeline: fetch PDF/API → parse → normalize → load SQLite
+1. `idx ownership import` — ETL pipeline: fetch XLSX/PDF → parse → normalize → load SQLite
 2. `idx ownership sync` — install a maintained SQLite snapshot via manifest + checksum validation
 3. All query commands read from local `~/.local/share/idx/ownership.db`
 4. Fully offline after import/sync
@ -81,7 +88,9 @@ Unlike the `stocks` module (live-fetch), ownership is **import-then-query**:
 ### Data Sources
 - **KSEI** — official ≥1% shareholder registry, published monthly by IDX. Through May 2026 it was a PDF announcement; from the 2026-05-29 report onward it is an XLSX workbook on the IDX "Data Kepemilikan Saham" page
 - **KSEI archive** — monthly ZIP/TXT balance-position matrix, used as a local fallback/backstop import path
- **Bing Finance** — global institutional ownership (REST API, quarterly)
+- **Bing Finance** — global institutional ownership. The schema, `--source bing`
  queries, and `src/api/msn/bing.rs` exist, but `ownership import --fetch-bing`
  is intentionally unsupported, so no ingest path populates Bing rows today
 ### Discovery
 `idx ownership discover` merges two IDX sources:
@ -123,7 +132,7 @@ CLI args → resolve symbol → provider.quote/history/fundamentals → render t
 ### Import-Query (ownership module)
 ```
-Import: PDF/API → parse → normalize → resolve entities → SQLite INSERT
+Import: XLSX/PDF/archive → parse → normalize → resolve entities → SQLite INSERT
 Query:  CLI args → SQLite SELECT → render table/json (no network)
 ```
--- a/docs/CONVENTIONS.md
+++ b/docs/CONVENTIONS.md
@ -126,7 +126,6 @@ let conn = Connection::open(path)
 - **Coverage over counts:** replacing redundant unit assertions with stronger integration scenarios must retain meaningful error, boundary, and data-integrity coverage.
 ## Git / VCS
 - **jj (Jujutsu)** as local workflow, colocated with git
 - Push via `nix develop --command git push` (for prek hooks)
- Branch naming: `feat/<name>`, `fix/<name>`
+- Branch naming: `feat/<name>`, `fix/<name>`, `docs/<name>`
 - Commit messages: conventional commits (`feat:`, `fix:`, `refactor:`, `docs:`)
--- a/docs/OWNERSHIP_PUBLISH.md
+++ b/docs/OWNERSHIP_PUBLISH.md
@ -1,170 +1,148 @@
 # Ownership Snapshot Publishing
-This document covers the maintainer workflow for producing and publishing the
+Maintainer workflow for producing and publishing the snapshot that
-ownership snapshot artifacts consumed by `idx ownership sync`.
+`idx ownership sync` consumes. The consumer contract (manifest schema and
 replacement rules) is in `docs/OWNERSHIP_SYNC.md`.
-## Goals
+## Release Layout
- Build the snapshot from the currently discoverable supported IDX/KSEI source,
+Snapshot assets live on a dedicated GitHub release, separate from versioned app
-  not from an ad hoc local DB.
+releases:
 - Keep the published manifest URL stable for end users.
 - Record source provenance in the manifest so the published SQLite artifact can
  be traced back to the live IDX announcement and PDF URL used to build it.
 ## Recommended GitHub Releases Layout
 Use a dedicated stable release tag for snapshot assets:
 - tag: `ownership-snapshot-current`
- manifest asset: `ownership-snapshot-manifest.json`
+- manifest: `ownership-snapshot-manifest.json` (stable URL, the commit point)
- SQLite asset: `ownership-snapshot-YYYY-MM-DD-<sha256>.sqlite` (immutable, content-addressed)
+- SQLite: `ownership-snapshot-YYYY-MM-DD-<sha256>.sqlite` (immutable, content-addressed)
 Recommended public manifest URL:
 ```text
 https://github.com/0xrsydn/idx-cli/releases/download/ownership-snapshot-current/ownership-snapshot-manifest.json
 ```
-Avoid `.../releases/latest/download/...` if normal app releases and snapshot
+Never use `.../releases/latest/download/...`: the repo's latest release is an
-publishes share the same repository. The repo's "latest" release can drift away
+app release, not the snapshot.
 from the ownership snapshot release.
-## Manual Maintainer Flow
+## Running the Publisher
-Run the publisher helper inside `nix develop` so `mutool` and the
+Packaged (recommended for hosts; no checkout or cargo build needed):
-`curl-impersonate` helper are available:
+
 ```bash
 nix build github:0xrsydn/idx-cli#ownership-publisher
 ./result/bin/idx-ownership-publish --output-dir /var/lib/idx/ownership-snapshot/current --history 2
 ./result/bin/idx-ownership-freshness --max-age-days 40
 ```
 From a checkout:
 ```bash
 nix develop --command cargo build
 nix develop --command scripts/publish-ownership-snapshot.sh \
  --build \
  --idx-bin ./target/debug/idx \
  --output-dir dist/ownership-snapshot \
  --repo 0xrsydn/idx-cli \
  --release-tag ownership-snapshot-current
 ```
-The helper performs these steps:
+`idx-ownership-publish` wraps `scripts/publish-ownership-snapshot.sh` with a
 pinned `idx` (including curl-impersonate and mupdf) plus jq, gh, sqlite, and
 curl. It needs `GH_TOKEN` with permission to upload release assets; the freshness
 check needs no token.
 ## What It Does
 1. `idx -o json ownership discover --family above1 --limit 50` (newest supported
   report first; XLSX from the Data Kepemilikan Saham page since June 2026)
-2. verifies the discovered report is the current supported import path
+2. verifies the discovered report is the supported import path
-3. imports that report (plus `--history <n>` earlier months, oldest first) into
+3. imports it plus `--history <n>` earlier months, oldest first, into an
-   an isolated temp ownership DB
+   isolated temp DB
-4. checks that the imported release metadata is non-empty and tied to the same
+4. checks the imported release metadata is non-empty and tied to the source URL
-   source URL
+5. builds the SQLite artifact and manifest (`scripts/build-latest-ownership-snapshot.sh`,
-5. runs `scripts/build-ownership-snapshot.sh` to emit the SQLite artifact and
+   which calls `scripts/build-ownership-snapshot.sh`) and records `source`
-   base manifest
+   provenance in the manifest
-6. enriches the manifest with `source` provenance metadata
+6. uploads using the protocol below
 The resulting manifest records:
 - `source.family`
 - `source.listing_page_url`
 - `source.query_url`
 - `source.pdf_url`
 - `source.title`
 - `source.publish_date`
 - `source.original_filename`
 That metadata is additive. Existing sync clients can still parse the manifest.
 ## Publication Protocol
-The manifest is the commit point that `idx ownership sync` follows. The
+The manifest is the commit point, so uploads are ordered:
 publisher therefore never uploads the manifest and the SQLite asset in one
 non-atomic call. It uses this order:
-1. Build the snapshot and stage the manifest plus one content-addressed SQLite
+1. Upload the content-addressed SQLite asset. Same content means same name, so
-   file.
+   retries are idempotent.
-2. Upload the SQLite asset first. The name includes the artifact's SHA-256, so
+2. Read the release assets back and verify the full SHA-256 and size (GitHub's
-   the same content always has the same name and a retry is idempotent.
+   `sha256:` digest when present, otherwise download and hash). Fail before
-3. Read the release assets back and confirm the SQLite asset exists with the
+   touching the manifest if this does not match.
-   expected SHA-256 and size. Use GitHub's `sha256:` digest metadata when it is
+3. Upload the manifest last and verify its `snapshot.sqlite_sha256`.
   present; for legacy assets without a digest, download the asset through `gh`
   and hash it. Fail before touching the manifest if verification fails.
 4. Upload the manifest last and verify that its `snapshot.sqlite_sha256`
   matches the staged manifest.
-Consequences:
+A partial upload exits non-zero and is repaired by the next run; previously
 published manifests stay valid because old SQLite assets are retained. GitHub
 does not replace the manifest transactionally, so a failed replacement can leave
 it unavailable until a retry succeeds.
- A partial upload (manifest uploaded, SQLite missing) can no longer happen.
+The publisher treats the published snapshot as current only when the manifest's
- If either upload fails, the job exits non-zero and the next run retries.
+`download_url` names this repo and tag and the referenced asset matches its
-  Previously downloaded manifests remain valid because the old SQLite asset is retained.
+SHA-256 and size. A missing or corrupt asset, a mismatched URL, an unreadable
- GitHub does not replace the manifest transactionally. A failed replacement
+manifest, or any GitHub auth/network error means "publish", never "up to date".
-  can leave the manifest unavailable until a retry succeeds.
+`--force` uploads regardless.
- The publisher treats a published manifest as current only when `download_url`
+
-  names this repo and release tag and the referenced asset matches the manifest
+## Run Result
-  SHA-256 and size. A missing or corrupt asset, a mismatched URL, or any GitHub
+
-  authentication/network error means "publish", never "up to date".
+The publisher is idempotent and prints exactly one final line:
 | Last line | Exit | Meaning |
 | --- | --- | --- |
 | `RESULT: published <as-of>` | 0 | a newer snapshot, or larger history, was uploaded |
 | `RESULT: up-to-date <as-of>` | 0 | nothing to upload |
 | `RESULT: FAILED stage=<stage> (exit N)` | N | failed in `arguments`, `build`, `preflight`, `discover`, `build-snapshot`, `stage-output`, `upload`, or `publish` |
 `idx-ownership-freshness` is the independent alarm: it exits 1 when the published
 `latest_as_of_date` is older than `--max-age-days` (default 40). The source lands
 2-3 days after month end, so 40 days only trips when publishing has stopped.
 ## History Coverage
-`--history <n>` requests the latest report plus up to `<n>` earlier monthly
+`--history <n>` (maximum 1000) requests the latest report plus up to `n` earlier
-XLSX reports. The publisher compares the request with the months the source
+monthly XLSX reports:
 actually exposes:
 - desired releases = `1 + min(--history, available earlier XLSX months)`
- a published snapshot with the same as-of date and at least the desired number
+- same as-of date with at least the desired release count: no-op
-  of releases is a no-op
+- same as-of date with fewer releases: republish to add history
- a published snapshot with the same as-of date but fewer releases is
+- a request larger than the source's history is accepted once, not rebuilt forever
  republished to add history
 - when the source has fewer historical months than requested, the publisher
  accepts the smaller coverage and does not rebuild on every run
-## Upload Step
+Each extra month adds about 2.4 MB, and history is what lets
 `idx ownership changes` work straight from a synced snapshot.
-Prefer the publisher helper, which implements the protocol above:
+## Snapshot Size Limit
-```bash
+idx clients up to v0.2.3 cap the snapshot download at **10 MiB** (10,485,760
-scripts/publish-ownership-snapshot.sh \
+bytes) and accept only uncompressed SQLite. A larger snapshot breaks
-  --idx-bin ./target/debug/idx \
+`idx ownership sync` for every existing install (this happened on 2026-09-25
-  --output-dir dist/ownership-snapshot \
+with `--history 5`).
  --repo 0xrsydn/idx-cli \
  --release-tag ownership-snapshot-current
 ```
-If you upload by hand, upload these files to the `ownership-snapshot-current`
+| `--history` | months | size (2026-08-31) |
-GitHub release in this order:
+| --- | --- | --- |
 | 2 | 3 | 8,171,520 |
 | 3 | 4 | 10,510,336 (too large) |
 | 5 | 6 | 15,290,368 (too large) |
-1. `dist/ownership-snapshot/ownership-snapshot-YYYY-MM-DD-<sha256>.sqlite`
+The builder refuses snapshots above `--max-snapshot-bytes` (default 10485760),
-2. `dist/ownership-snapshot/ownership-snapshot-manifest.json`
+ending with `RESULT: FAILED stage=build-snapshot` and leaving the published
 snapshot unchanged. Newer clients accept up to the manifest's `size_bytes` (at
 most 1 GiB); raise both limits only once pre-v0.2.4 clients are rare.
-Never upload the manifest before its SQLite asset is present. Only after this
+## Automation
 manual flow is reliable should the repo automate it in GitHub Actions.
-## GitHub Actions Workflow
+IDX has returned `403` to GitHub-hosted runners, so unattended publishing runs
 on a self-hosted machine that IDX accepts:
-The repo now includes a manual workflow at
+- `contrib/systemd/idx-ownership-snapshot-publish.service` and `.timer` run the
-`.github/workflows/publish-ownership-snapshot.yml`.
+  packaged publisher daily at 06:00 UTC with `GH_TOKEN` from
  `/etc/idx-ownership-snapshot.env`. Wire `OnFailure=` to a notification.
 - On NixOS, add this repo as a flake input and use
  `inputs.idx-cli.packages.${system}.ownership-publisher`; `nix flake update idx-cli`
  picks up fixes.
-Current behavior:
+`.github/workflows/publish-ownership-snapshot.yml` is a manual
 (`workflow_dispatch`) fallback that runs the same packaged publisher with inputs
 passed as environment variables.
- trigger: `workflow_dispatch` only
+Observed XLSX upload times (HTTP Last-Modified) were the 2nd-3rd of each month
- builds the packaged publisher with `nix build .#ownership-publisher`
+around 02:00-04:00 UTC; a daily idempotent run makes the exact day irrelevant.
- runs `idx-ownership-publish` from that package, which performs discovery,
+If the monthly source has not appeared yet, the run should fail loudly and retry
-  import, snapshot build, and the full upload protocol in one pass
+later rather than publish stale data.
 - uploads any locally produced artifacts as a workflow artifact copy with
  `if-no-files-found: ignore`, so a legitimate up-to-date run still succeeds
 - does not run a separate release-create or asset-upload step
 This is intentionally manual-first. Add a schedule only after a few successful
 publish runs confirm the live source remains stable enough.
 The workflow passes its inputs to the shell through environment variables and
 routes publication through the same fixed publisher used by self-hosted hosts,
 so it cannot drift from the CLI protocol.
 ## Self-Hosted Automation
 GitHub-hosted Actions are still not sufficient for this job on their own. Live
 verification has shown IDX returning `403` to GitHub-hosted runners during
 discovery/import.
 The recommended unattended path is therefore:
 - keep the public artifacts on GitHub Releases
 - run the publish job from a self-hosted machine that IDX accepts
 - use a `systemd` oneshot service plus `systemd.timer` on that machine
 See `docs/OWNERSHIP_SELF_HOSTED.md` for the reusable helper script, sample
 systemd units, and the recommended split between this repo and `clan-private`.
--- a/docs/OWNERSHIP_SELF_HOSTED.md
+++ b/docs/OWNERSHIP_SELF_HOSTED.md
@ -1,149 +0,0 @@
 # Self-Hosted Ownership Snapshot Publish
 This document covers the recommended self-hosted automation path for publishing
 the monthly ownership snapshot consumed by `idx ownership sync`.
 Use this path when:
 - the repo's public release assets should stay on GitHub Releases
 - IDX blocks GitHub-hosted Actions with `403`
 - you have a VPS or other trusted machine that can reach the IDX source
 ## Why Self-Hosted
 The current GitHub-hosted workflow remains useful as a manual reference, but it
 is not reliable enough for unattended publishing because IDX has returned `403`
 to GitHub-hosted runners. The self-hosted path keeps the end-user experience
 simple:
 - maintainers publish from a trusted machine
 - end users still run plain `idx ownership sync`
 No Nix setup is required for end users. Nix is only an implementation detail for
 the maintainer host if that host already uses NixOS.
 ## Packaged Publisher (recommended)
 The flake exposes a self-contained publisher, so the host needs no checkout,
 `nix develop`, or cargo build at run time:
 ```bash
 nix build github:0xrsydn/idx-cli#ownership-publisher
 ./result/bin/idx-ownership-publish --output-dir /var/lib/idx/ownership-snapshot/current --history 5
 ./result/bin/idx-ownership-freshness --max-age-days 40
 ```
 The publisher needs `GH_TOKEN` in its environment: a token that can create
 and upload assets on the target repo's releases (the examples read it from
 `/etc/idx-ownership-snapshot.env`; clan-private uses a Clan var). The
 freshness check needs no token.
 `idx-ownership-publish` wraps `scripts/publish-ownership-snapshot.sh` with a
 pinned `idx` (itself wrapped with curl-impersonate and mupdf) plus jq, gh,
 sqlite and curl. On NixOS, add this repo as a flake input and run
 `inputs.idx-cli.packages.${system}.ownership-publisher`; bump the input
 (`nix flake update idx-cli`) to pick up fixes.
 ## Run Semantics
 The publisher is idempotent and prints exactly one final line:
 | Last line | Exit | Meaning |
 | --- | --- | --- |
 | `RESULT: published <as-of>` | 0 | a newer snapshot, or a larger history, was uploaded |
 | `RESULT: up-to-date <as-of>` | 0 | the published snapshot already has the latest report and requested history; nothing uploaded |
 | `RESULT: FAILED stage=<stage> (exit N)` | N | failed in `arguments`, `build`, `preflight`, `discover`, `build-snapshot`, `stage-output`, `upload`, or `publish` |
 It checks the published manifest and verifies that the manifest's `download_url`
 names this repo and release tag, and that the referenced SQLite asset matches
 the manifest SHA-256 and size, before it skips anything. It uses GitHub's
 `sha256:` digest metadata when available and downloads legacy assets without a
 digest to hash them. A missing or corrupt asset, a mismatched URL, an unreadable
 manifest, or a GitHub authentication/network error is treated as "publish",
 never as up to date. The upload path verifies the full SHA-256 of the uploaded
 asset before it publishes the manifest.
 For XLSX sources it also compares the requested `--history` coverage with the
 months the source actually exposes, so a history increase is published once and
 a request larger than the available history does not rebuild forever. `--history`
 is bounded (maximum 1000) so an enormous value cannot wrap and look already
 satisfied. The checks use discovery only, so a daily run stays cheap. `--force`
 uploads regardless.
 The SQLite asset is immutable and content-addressed. The publisher uploads and
 verifies it before it uploads the manifest, which is the commit point consumers
 follow. A partial upload therefore exits non-zero and is repaired by the next
 run.
 `idx-ownership-freshness` is the independent alarm: it exits 1 when the
 published `latest_as_of_date` is older than `--max-age-days` (default 40). The
 source is monthly and lands 2-3 days after month end, so 40 days only trips
 when publishing has actually stopped working. Wire both to an `OnFailure=`
 notification.
 ## Manual Command (from a checkout)
 ```bash
 nix develop --command scripts/publish-ownership-snapshot.sh \
  --build \
  --idx-bin ./target/debug/idx \
  --output-dir /var/lib/idx/ownership-snapshot/current \
  --repo 0xrsydn/idx-cli \
  --release-tag ownership-snapshot-current
 ```
 ## systemd Units
 Examples: `contrib/systemd/idx-ownership-snapshot-publish.service` and
 `contrib/systemd/idx-ownership-snapshot-publish.timer` (daily at 06:00 UTC).
 Observed XLSX upload times (HTTP Last-Modified): 2026-06-03,
 2026-07-02 03:55Z, 2026-08-02 01:50Z, 2026-09-02 03:51Z. With a daily,
 idempotent run the exact day no longer matters.
 Pass `--history <n>` to include earlier months so `idx ownership changes`
 works straight from the synced snapshot (about 2.4 MB per extra month).
 ### Snapshot size limit
 idx clients up to v0.2.3 cap the snapshot download at **10 MiB** (10,485,760
 bytes) and only accept uncompressed SQLite. A larger snapshot makes
 `idx ownership sync` fail for every existing install. This happened on
 2026-09-25 with `--history 5` (15.3 MB).
 | `--history` | months | size (2026-08-31) |
 | --- | --- | --- |
 | 2 | 3 | 8,171,520 |
 | 3 | 4 | 10,510,336 (too large) |
 | 5 | 6 | 15,290,368 (too large) |
 The builder therefore refuses snapshots above `--max-snapshot-bytes`
 (default 10485760). The run ends `RESULT: FAILED stage=build-snapshot` and the
 published snapshot stays unchanged. Newer clients accept snapshots up to the
 manifest's `size_bytes` (at most 1 GiB, buffered in memory);
 raise `--max-snapshot-bytes` and `--history` only after clients without the
 10 MiB cap are widespread.
 ## Clan Integration Later
 The clean split for `clan-private` is:
 1. keep the workflow logic in this repo
 2. keep host-specific secrets and enablement in `clan-private`
 Recommended integration shape on `greencloud-vps`:
 - store a checked-out copy of `idx-cli` on the host, for example at `/srv/idx-cli`
 - provide `GH_TOKEN` through Clan vars or another secret mechanism
 - add a small NixOS module that installs/enables the service and timer
 - set the module's `WorkingDirectory`, env file path, and publish directory
 The existing pattern in `clan-private/modules/workspace-backup.nix` is a good
 fit: a oneshot `systemd` service plus a `systemd.timer`.
 ## Operational Notes
 - Keep `ownership-snapshot-current` separate from versioned app releases.
 - Prefer manual runs first, then enable the timer after a few successful
  publishes.
 - If the monthly source has not appeared yet, the service should fail loudly and
  be retried later; do not silently publish stale assumptions.
--- a/docs/OWNERSHIP_SYNC.md
+++ b/docs/OWNERSHIP_SYNC.md
@ -4,8 +4,7 @@
 The command is intentionally manifest-driven so the repo can publish snapshots in GitHub releases, object storage, or a local filesystem path without changing the CLI.
-The recommended GitHub publish layout and maintainer workflow are documented in
+Publishing snapshots is a maintainer task; see `docs/OWNERSHIP_PUBLISH.md`.
 `docs/OWNERSHIP_PUBLISH.md`.
 ## Consumer Inputs
@ -55,7 +54,8 @@ Current schema version: `1`
 ```
 Semantics:
- `source` is optional provenance metadata describing the IDX/KSEI PDF used to build the snapshot.
+- `source` is optional provenance metadata describing the IDX/KSEI report used to build the snapshot.
  `source.pdf_url` is a legacy field name: since June 2026 it holds the XLSX URL.
 - `download_url` points to the SQLite artifact itself. The filename is an
  implementation detail: published assets are immutable and content-addressed
  (`ownership-snapshot-YYYY-MM-DD-<sha256>.sqlite`), so consumers must always
@ -76,20 +76,3 @@ Without `--force`, sync behaves conservatively:
 - local DB diverges at the same latest date: no-op and require `--force`
 Replacement is staged through a temp file and validated before the local DB is swapped in.
 ## Publisher Workflow
 Use `scripts/build-ownership-snapshot.sh` to copy a vetted ownership DB into a publishable artifact plus manifest:
 ```bash
 scripts/build-ownership-snapshot.sh \
  --db /path/to/ownership.db \
  --output-dir dist/ownership-snapshot \
  --base-url https://example.com/idx/ownership
 ```
 If `--base-url` is omitted, the generated manifest uses the local artifact path as `download_url`, which is useful for local testing.
 For the maintainer flow that discovers the latest supported IDX/KSEI source
 first and then builds GitHub-release-ready assets, use
 `scripts/build-latest-ownership-snapshot.sh`.
--- a/docs/SMOKE.md
+++ b/docs/SMOKE.md
@ -105,8 +105,6 @@ responses honor the requested batch size so the complete-fetch path is exercised
 Mocks do not verify current upstream availability, authentication, or response
 drift. Live provider checks remain a separate, opt-in operation.
 ## Modes
 - `live` runs the default real-network baseline: `general`, `live-table`, and `ownership`
@ -128,12 +126,10 @@ drift. Live provider checks remain a separate, opt-in operation.
 ## Notes
- The runner forces `IDX_OUTPUT=table` as its default environment so table cases stay stable; JSON checks use `-o json` explicitly.
+- The runner forces `IDX_OUTPUT=table` so table cases stay stable; JSON checks pass `-o json` explicitly.
- Cache-group warm cases clear the smoke cache before they run so each warm/offline/stale sequence starts clean and stale-cache assertions are not masked by earlier groups.
+- Cache-group warm cases clear the smoke cache first so warm/offline/stale sequences start clean.
- Use `--bin <path> --no-build` when you want to validate an installed binary instead of the workspace `target/debug/idx` build.
+- Use `--bin <path> --no-build` to validate an installed binary instead of `target/debug/idx`.
- Use `scripts/audit-msn-fundamentals.sh` for a full CLI valuation sweep across the IDX MSN symbol map. It is intentionally separate from the reusable smoke runner because it is a heavier provider-health audit, not a stable baseline check.
+- `ownership-import` and `live-nonfinite` are opt-in (explicit `--group` or `--mode full`). Known non-finite repro tickers as of `2026-04-13`: `BUMI`, `ADRO`, `AIMS`.
- Ownership commands that need imported data are intentionally not part of the baseline runner yet. The current baseline only covers `ownership releases` and the known unsupported `ownership import --fetch-bing`.
+- The baseline `ownership` group needs no imported data; `ownership sync` is covered by fixture-backed CLI tests instead.
- `ownership sync` is still primarily covered by fixture-backed CLI tests rather than the reusable smoke runner.
+- `scripts/audit-msn-fundamentals.sh` is a heavier provider-health sweep over the IDX MSN symbol map, not a baseline check.
- The new `ownership-import` group is intentionally opt-in for explicit `--group ownership-import` runs or `--mode full`; it discovers live URLs first, imports the supported `above1` attachment into the temp DB, then asserts the current `above5` and `investor-type` URLs fail with explicit unsupported-schema UX.
+- When a case fails, inspect its `.stdout`, `.stderr`, and `.validation` files under `tmp/live-smoke/.../logs/`.
 - The new `live-nonfinite` group is intentionally opt-in only. As of `2026-04-13`, the known real repro tickers are `BUMI`, `ADRO`, and `AIMS`.
 - When a case fails, inspect its `.stdout`, `.stderr`, and (for JSON cases) `.validation` files in `tmp/live-smoke/.../logs/` before updating `TODO.md` or `FEATURE_SPEC.md`.