mirror of
https://github.com/0xrsydn/idx-cli.git
synced 2026-08-07 01:33:52 +00:00
fix: handle missing scrip column in April 2026 KSEI PDF
The April 2026 IDX ownership PDF omits the zero-valued scrip column for some holders, causing the parser to leave holdings_scripless/scrip as empty strings. This broke normalize_ksei_row which called parse_id_number on empty input. Two-layer fix: - Parser: pop_numeric_tail handles 2-column (missing scrip) rows by defaulting scrip to "0" - Normalizer: parse_id_number_or_zero backstop treats empty component share fields as 0 while still requiring total_shares Also includes AGENTS.md refresh, formatting cleanup (rustfmt), and version bump to 0.2.2. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
c94c826307
commit
04eaa75882
10 changed files with 286 additions and 144 deletions
159
AGENTS.md
159
AGENTS.md
|
|
@ -1,95 +1,102 @@
|
||||||
# AGENTS.md
|
# AGENTS.md
|
||||||
|
|
||||||
## Project
|
## Project
|
||||||
`idx-cli` — CLI tool for Indonesian stock market (IDX) analysis. Built in Rust for humans and AI agents. Single binary, schema-driven, functional architecture.
|
`idx-cli` is a Rust CLI for Indonesian stock market analysis and ownership workflows.
|
||||||
|
|
||||||
|
The repo currently has two main product areas:
|
||||||
|
- `stocks`: live market data and analysis
|
||||||
|
- `ownership`: import/sync once, then query locally from SQLite
|
||||||
|
|
||||||
## Stack
|
## Stack
|
||||||
- **Language:** Rust (stable, via rust-overlay)
|
- Rust stable
|
||||||
- **CLI:** clap 4 (derive)
|
- `clap` 4 for CLI
|
||||||
- **HTTP:** ureq 3 (sync, no async runtime)
|
- `ureq` 3 for HTTP
|
||||||
- **Output:** comfy-table, owo-colors
|
- `comfy-table` and `owo-colors` for output
|
||||||
- **DB:** rusqlite (bundled SQLite, FTS5) — ownership module
|
- `rusqlite` + bundled SQLite/FTS5 for ownership
|
||||||
- **Config:** TOML (`~/.config/idx/config.toml`)
|
- TOML config in `~/.config/idx/config.toml`
|
||||||
- **Cache:** JSON file-based (`~/.cache/idx/`)
|
- file cache in `~/.cache/idx/`
|
||||||
- **Testing:** cargo test, assert_cmd, predicates
|
|
||||||
- **Hooks:** prek (pre-commit: fmt+clippy, pre-push: test)
|
|
||||||
- **VCS:** jj (Jujutsu, colocated with git)
|
|
||||||
|
|
||||||
## Structure
|
## Source Map
|
||||||
```
|
```text
|
||||||
src/
|
src/
|
||||||
├── main.rs # Entry point, command dispatch
|
├── main.rs
|
||||||
├── cli/ # Command handlers (clap derive structs)
|
├── cli/
|
||||||
│ ├── stocks.rs # stocks quote/history/technical/fundamental/...
|
│ ├── stocks.rs
|
||||||
│ ├── config.rs # config get/set/init/path
|
│ ├── ownership.rs
|
||||||
│ ├── cache.rs # cache info/clear
|
│ ├── config.rs
|
||||||
│ └── ownership.rs # ownership import/query commands
|
│ └── cache.rs
|
||||||
├── api/ # Data providers (trait-based abstraction)
|
├── api/
|
||||||
│ ├── mod.rs # MarketDataProvider trait + factory functions
|
│ ├── mod.rs
|
||||||
│ ├── types.rs # All domain types (Quote, Ohlc, Fundamentals, ...)
|
│ ├── types.rs
|
||||||
│ ├── yahoo/ # Yahoo Finance provider (history/OHLCV)
|
│ ├── yahoo/
|
||||||
│ └── msn/ # MSN Finance provider (quotes, fundamentals, ++)
|
│ └── msn/
|
||||||
├── analysis/ # Technical & fundamental analysis (pure functions)
|
├── analysis/
|
||||||
├── ownership/ # Ownership intelligence module (SQLite-backed)
|
├── ownership/
|
||||||
│ ├── types.rs # Ownership domain types
|
│ ├── archive.rs
|
||||||
│ └── db.rs # Schema, migrations, queries
|
│ ├── db.rs
|
||||||
├── output/ # Rendering (table, json)
|
│ ├── entities.rs
|
||||||
├── cache.rs # File-based TTL cache
|
│ ├── parser.rs
|
||||||
├── config.rs # Config loading (flags > env > file > defaults)
|
│ ├── remote.rs
|
||||||
└── error.rs # IdxError enum (thiserror)
|
│ ├── snapshot.rs
|
||||||
|
│ └── types.rs
|
||||||
|
├── output/
|
||||||
|
├── cache.rs
|
||||||
|
├── config.rs
|
||||||
|
└── error.rs
|
||||||
```
|
```
|
||||||
|
|
||||||
## Providers
|
## Provider Model
|
||||||
- **MSN** = default provider (quotes, fundamentals, profile, earnings, financials, sentiment, insights, news, screener)
|
- `MSN` is the primary/default provider for quotes, fundamentals, profile, earnings, financials, sentiment, insights, news, and screener data.
|
||||||
- **Yahoo** = automatic fallback for history/OHLCV (MSN doesn't support IDX history)
|
- `Yahoo` is the fallback provider for history/OHLCV because MSN history for IDX is still not supported.
|
||||||
- Configurable: `IDX_PROVIDER=msn|yahoo`, `IDX_HISTORY_PROVIDER=auto|yahoo|msn`
|
- Current config knobs:
|
||||||
|
- `IDX_PROVIDER=msn|yahoo`
|
||||||
|
- `IDX_HISTORY_PROVIDER=auto|yahoo|msn`
|
||||||
|
|
||||||
## Current Status
|
## Ownership Model
|
||||||
- Automated coverage is healthy: `cargo test` currently passes with 122 tests (86 unit, 36 integration).
|
Ownership is local-first after bootstrap.
|
||||||
- Live `stocks` commands are implemented and smoke-tested for: `quote`, `history`, `technical`, `growth`, `valuation`, `risk`, `fundamental`, `compare`, `profile`, `financials`, `earnings`, `sentiment`, `insights`, `news`, `screen`.
|
|
||||||
- `stocks history --history-provider msn` is intentionally unsupported for IDX; `auto` falls back to Yahoo.
|
|
||||||
- `ownership import --fetch-bing` is still intentionally unsupported; Bing client groundwork exists but the CLI path is deferred.
|
|
||||||
|
|
||||||
## Known Hardening Gaps
|
Preferred bootstrap/update path:
|
||||||
- MSN-only commands still bypass the shared cache/offline path in `src/cli/stocks.rs`; `--offline` is not reliable for `profile`/`financials`/`earnings`/`sentiment`/`insights`/`news`/`screen`.
|
1. `idx ownership sync`
|
||||||
- Core quote flow has a verified `--offline --no-cache` bug: stale cache can still be served.
|
2. if no snapshot manifest is available: `idx ownership discover` then `idx ownership import --url <pdf-url>`
|
||||||
- Startup/config failures do not yet honor the JSON error contract; runtime failures do.
|
3. local `--file` imports remain available for manual/fallback use
|
||||||
- `stocks screen --filter` and `--region` still silently coerce invalid values instead of rejecting them.
|
|
||||||
- Some live MSN output is incomplete or misleading:
|
|
||||||
- `profile` can return sparse fields.
|
|
||||||
- `insights.last_updated` is still empty.
|
|
||||||
- `financials` table output has malformed negative-number formatting in some rows.
|
|
||||||
|
|
||||||
## Development
|
Ownership input paths:
|
||||||
```bash
|
- primary remote source: discoverable IDX `above1` holder-register PDF
|
||||||
nix develop # enter dev shell
|
- maintained snapshot path: `ownership sync`
|
||||||
cargo build # build
|
- local fallback path: PDF, plus local archive `.zip` / `.txt`
|
||||||
cargo run -- stocks quote BBCA # run
|
|
||||||
cargo run -- -o json stocks history BBCA # JSON output
|
Important scope note:
|
||||||
cargo test # test
|
- archive ZIP/TXT ingest is a fallback/backstop path, not the primary product ingest surface
|
||||||
cargo fmt --check && cargo clippy -- -D warnings # lint
|
- `ownership import --fetch-bing` is still intentionally unsupported
|
||||||
```
|
|
||||||
|
## Working Principles
|
||||||
|
1. Keep data access provider-driven where possible; avoid adding new ad hoc fetch paths at the CLI layer.
|
||||||
|
2. Prefer pure parse/normalize transforms over hidden state.
|
||||||
|
3. Use fixtures in tests; do not hit live network in automated tests.
|
||||||
|
4. Preserve the output contract: table to stdout, JSON with `--output json`, errors to stderr.
|
||||||
|
5. Treat ownership schema/query compatibility as important: `releases`, `ticker`, and `changes` should keep working across ingest paths.
|
||||||
|
|
||||||
## Verification
|
## Verification
|
||||||
|
Core verification:
|
||||||
```bash
|
```bash
|
||||||
cargo build # must compile
|
nix develop
|
||||||
cargo clippy -- -D warnings # zero warnings
|
cargo build
|
||||||
cargo test # all tests pass
|
cargo clippy -- -D warnings
|
||||||
|
cargo test
|
||||||
```
|
```
|
||||||
|
|
||||||
## Principles
|
Smoke tooling:
|
||||||
1. **Schema-driven** — define types first, build logic around them. Types are the spec.
|
```bash
|
||||||
2. **Functional approach** — pure parse/transform functions (`parse_*`, `normalize_*`), no hidden state.
|
scripts/live-smoke.sh
|
||||||
3. **Data types heavy** — rich enums, newtypes, composite structs. Precision via integer representations (basis points for %, i64 for shares).
|
scripts/live-smoke.sh --mode mock
|
||||||
4. **Provider abstraction first** — all data access should flow through traits/factories. Note: current MSN-only stock commands still instantiate `MsnProvider` directly in `src/cli/stocks.rs`; removing that split path is an active hardening target.
|
scripts/live-smoke.sh --mode full
|
||||||
5. **Sync only** — no tokio/async. CLI tool, ureq is sufficient.
|
```
|
||||||
6. **Test with fixtures** — never hit live APIs in tests. Mock provider + fixture JSON.
|
|
||||||
7. **Output contract** — table to stdout (humans), `--output json` (machines), errors to stderr.
|
|
||||||
8. **Feature-gated modules** — `ownership` feature for SQLite dep, keeps base binary lean.
|
|
||||||
|
|
||||||
## Docs
|
## Read First
|
||||||
Start with the repo-visible docs:
|
Start with these repo docs before making changes:
|
||||||
- `FEATURE_SPEC.md` — current hardening backlog and CLI truth-pass expectations
|
- `FEATURE_SPEC.md` — active implementation backlog and remaining core gaps
|
||||||
- `TODO.md` — working task list, including latest smoke findings
|
- `TODO.md` — execution tracker and smoke notes
|
||||||
- `docs/ARCHITECTURE.md` — provider/capability design and error strategy
|
- `docs/ARCHITECTURE.md` — provider and ownership flow
|
||||||
|
- `docs/OWNERSHIP_SYNC.md` — snapshot sync contract
|
||||||
|
- `docs/SMOKE.md` — reusable smoke commands
|
||||||
- `docs/CONVENTIONS.md` — repo conventions
|
- `docs/CONVENTIONS.md` — repo conventions
|
||||||
|
|
|
||||||
2
Cargo.lock
generated
2
Cargo.lock
generated
|
|
@ -711,7 +711,7 @@ dependencies = [
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "idx-cli"
|
name = "idx-cli"
|
||||||
version = "0.2.1"
|
version = "0.2.2"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"assert_cmd",
|
"assert_cmd",
|
||||||
"chrono",
|
"chrono",
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
[package]
|
[package]
|
||||||
name = "idx-cli"
|
name = "idx-cli"
|
||||||
version = "0.2.1"
|
version = "0.2.2"
|
||||||
edition = "2024"
|
edition = "2024"
|
||||||
rust-version = "1.85"
|
rust-version = "1.85"
|
||||||
description = "CLI tool for Indonesian stock market (IDX) analysis"
|
description = "CLI tool for Indonesian stock market (IDX) analysis"
|
||||||
|
|
|
||||||
|
|
@ -14,11 +14,11 @@ use crate::error::IdxError;
|
||||||
use crate::output::OutputFormat;
|
use crate::output::OutputFormat;
|
||||||
use crate::output::json;
|
use crate::output::json;
|
||||||
use crate::output::table::format_idr;
|
use crate::output::table::format_idr;
|
||||||
use crate::runtime;
|
|
||||||
use crate::ownership::types::{
|
use crate::ownership::types::{
|
||||||
ChangeType, FlowSignal, HolderRow, KseiHolding, OwnershipRelease, OwnershipSource,
|
ChangeType, FlowSignal, HolderRow, KseiHolding, OwnershipRelease, OwnershipSource,
|
||||||
};
|
};
|
||||||
use crate::ownership::{archive, db, entities, graph, parser, remote, search, snapshot};
|
use crate::ownership::{archive, db, entities, graph, parser, remote, search, snapshot};
|
||||||
|
use crate::runtime;
|
||||||
|
|
||||||
#[derive(Debug, Args)]
|
#[derive(Debug, Args)]
|
||||||
#[command(
|
#[command(
|
||||||
|
|
|
||||||
18
src/main.rs
18
src/main.rs
|
|
@ -78,16 +78,14 @@ fn run() -> Result<(), IdxError> {
|
||||||
}
|
}
|
||||||
Commands::Stocks(stocks) => {
|
Commands::Stocks(stocks) => {
|
||||||
let provider = default_provider(config.provider, cli.verbose > 0);
|
let provider = default_provider(config.provider, cli.verbose > 0);
|
||||||
if let Err(err) =
|
if let Err(err) = cli::stocks::handle(
|
||||||
cli::stocks::handle(
|
stocks,
|
||||||
stocks,
|
&config,
|
||||||
&config,
|
&provider,
|
||||||
&provider,
|
cli.offline,
|
||||||
cli.offline,
|
cli.no_cache,
|
||||||
cli.no_cache,
|
cli.verbose > 0,
|
||||||
cli.verbose > 0,
|
) {
|
||||||
)
|
|
||||||
{
|
|
||||||
emit_error(&err, &config.output);
|
emit_error(&err, &config.output);
|
||||||
return Err(err);
|
return Err(err);
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -159,7 +159,10 @@ pub fn upsert_ticker(conn: &Connection, code: &str, name: Option<&str>) -> Resul
|
||||||
.map_err(|e| IdxError::DatabaseError(e.to_string()))
|
.map_err(|e| IdxError::DatabaseError(e.to_string()))
|
||||||
}
|
}
|
||||||
|
|
||||||
fn insert_ksei_holdings_rows(conn: &Connection, holdings: &[KseiHolding]) -> Result<usize, IdxError> {
|
fn insert_ksei_holdings_rows(
|
||||||
|
conn: &Connection,
|
||||||
|
holdings: &[KseiHolding],
|
||||||
|
) -> Result<usize, IdxError> {
|
||||||
if holdings.is_empty() {
|
if holdings.is_empty() {
|
||||||
return Ok(0);
|
return Ok(0);
|
||||||
}
|
}
|
||||||
|
|
@ -591,8 +594,10 @@ pub fn query_cross_holders(
|
||||||
min_tickers: usize,
|
min_tickers: usize,
|
||||||
limit: usize,
|
limit: usize,
|
||||||
) -> Result<Vec<CrossHolderRow>, IdxError> {
|
) -> Result<Vec<CrossHolderRow>, IdxError> {
|
||||||
let mut aggregates: std::collections::BTreeMap<i64, (Entity, std::collections::BTreeSet<i64>, i64)> =
|
let mut aggregates: std::collections::BTreeMap<
|
||||||
std::collections::BTreeMap::new();
|
i64,
|
||||||
|
(Entity, std::collections::BTreeSet<i64>, i64),
|
||||||
|
> = std::collections::BTreeMap::new();
|
||||||
|
|
||||||
if let Some(latest_ksei) = latest_ksei_release(conn)? {
|
if let Some(latest_ksei) = latest_ksei_release(conn)? {
|
||||||
let mut stmt = conn
|
let mut stmt = conn
|
||||||
|
|
@ -1437,7 +1442,10 @@ mod tests {
|
||||||
row_count: holdings.len(),
|
row_count: holdings.len(),
|
||||||
imported_at: 1,
|
imported_at: 1,
|
||||||
};
|
};
|
||||||
assert_eq!(write_ksei_release(&conn, &release, &holdings, false).unwrap(), 2);
|
assert_eq!(
|
||||||
|
write_ksei_release(&conn, &release, &holdings, false).unwrap(),
|
||||||
|
2
|
||||||
|
);
|
||||||
|
|
||||||
let data = query_ticker_holdings(&conn, "BBCA").unwrap();
|
let data = query_ticker_holdings(&conn, "BBCA").unwrap();
|
||||||
assert_eq!(data.ticker.code, "BBCA");
|
assert_eq!(data.ticker.code, "BBCA");
|
||||||
|
|
@ -1512,8 +1520,13 @@ mod tests {
|
||||||
};
|
};
|
||||||
|
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
write_ksei_release(&conn, &initial_release, std::slice::from_ref(&initial), false)
|
write_ksei_release(
|
||||||
.unwrap(),
|
&conn,
|
||||||
|
&initial_release,
|
||||||
|
std::slice::from_ref(&initial),
|
||||||
|
false
|
||||||
|
)
|
||||||
|
.unwrap(),
|
||||||
1
|
1
|
||||||
);
|
);
|
||||||
|
|
||||||
|
|
@ -1794,21 +1807,21 @@ mod tests {
|
||||||
let holdings = rows
|
let holdings = rows
|
||||||
.into_iter()
|
.into_iter()
|
||||||
.map(|(tid, bps, name)| KseiHolding {
|
.map(|(tid, bps, name)| KseiHolding {
|
||||||
id: 0,
|
id: 0,
|
||||||
ticker_id: tid,
|
ticker_id: tid,
|
||||||
entity_id: None,
|
entity_id: None,
|
||||||
raw_investor_name: name.to_string(),
|
raw_investor_name: name.to_string(),
|
||||||
investor_type: None,
|
investor_type: None,
|
||||||
locality: None,
|
locality: None,
|
||||||
nationality: None,
|
nationality: None,
|
||||||
domicile: None,
|
domicile: None,
|
||||||
holdings_scripless: 1,
|
holdings_scripless: 1,
|
||||||
holdings_scrip: 0,
|
holdings_scrip: 0,
|
||||||
total_shares: 1,
|
total_shares: 1,
|
||||||
percentage_bps: bps,
|
percentage_bps: bps,
|
||||||
report_date: NaiveDate::from_ymd_opt(2026, 2, 27).unwrap(),
|
report_date: NaiveDate::from_ymd_opt(2026, 2, 27).unwrap(),
|
||||||
release_sha256: "current-release".to_string(),
|
release_sha256: "current-release".to_string(),
|
||||||
})
|
})
|
||||||
.collect::<Vec<_>>();
|
.collect::<Vec<_>>();
|
||||||
let release = OwnershipRelease {
|
let release = OwnershipRelease {
|
||||||
id: 0,
|
id: 0,
|
||||||
|
|
|
||||||
|
|
@ -28,6 +28,14 @@ pub fn parse_id_number(s: &str) -> Result<i64, IdxError> {
|
||||||
.map_err(|e| IdxError::ParseError(format!("failed to parse Indonesian number '{s}': {e}")))
|
.map_err(|e| IdxError::ParseError(format!("failed to parse Indonesian number '{s}': {e}")))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn parse_id_number_or_zero(s: &str) -> Result<i64, IdxError> {
|
||||||
|
if s.trim().is_empty() {
|
||||||
|
Ok(0)
|
||||||
|
} else {
|
||||||
|
parse_id_number(s)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Parse Indonesian locale percentage to basis points (i64).
|
/// Parse Indonesian locale percentage to basis points (i64).
|
||||||
/// "54,94" → 5494, "0,00" → 0, "100,00" → 10000
|
/// "54,94" → 5494, "0,00" → 0, "100,00" → 10000
|
||||||
pub fn parse_id_percentage(s: &str) -> Result<i64, IdxError> {
|
pub fn parse_id_percentage(s: &str) -> Result<i64, IdxError> {
|
||||||
|
|
@ -141,8 +149,8 @@ pub fn normalize_ksei_row(raw: &KseiRawRow) -> Result<KseiHoldingDraft, IdxError
|
||||||
locality,
|
locality,
|
||||||
nationality: optional_string(&raw.nationality),
|
nationality: optional_string(&raw.nationality),
|
||||||
domicile: optional_string(&raw.domicile),
|
domicile: optional_string(&raw.domicile),
|
||||||
holdings_scripless: parse_id_number(&raw.holdings_scripless)?,
|
holdings_scripless: parse_id_number_or_zero(&raw.holdings_scripless)?,
|
||||||
holdings_scrip: parse_id_number(&raw.holdings_scrip)?,
|
holdings_scrip: parse_id_number_or_zero(&raw.holdings_scrip)?,
|
||||||
total_shares: parse_id_number(&raw.total_holding_shares)?,
|
total_shares: parse_id_number(&raw.total_holding_shares)?,
|
||||||
percentage_bps: parse_id_percentage(&raw.percentage)?,
|
percentage_bps: parse_id_percentage(&raw.percentage)?,
|
||||||
report_date: parse_ksei_date(&raw.date)?,
|
report_date: parse_ksei_date(&raw.date)?,
|
||||||
|
|
@ -297,9 +305,10 @@ mod tests {
|
||||||
use rusqlite::Connection;
|
use rusqlite::Connection;
|
||||||
|
|
||||||
use crate::ownership::entities::{
|
use crate::ownership::entities::{
|
||||||
normalize_name, parse_id_number, parse_id_percentage, parse_ksei_date, resolve_entity,
|
normalize_ksei_row, normalize_name, parse_id_number, parse_id_percentage, parse_ksei_date,
|
||||||
|
resolve_entity,
|
||||||
};
|
};
|
||||||
use crate::ownership::types::OwnershipSource;
|
use crate::ownership::types::{KseiRawRow, OwnershipSource};
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn test_parse_id_number() {
|
fn test_parse_id_number() {
|
||||||
|
|
@ -401,4 +410,47 @@ mod tests {
|
||||||
|
|
||||||
assert_eq!(id1, id2);
|
assert_eq!(id1, id2);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn test_normalize_ksei_row_defaults_empty_component_shares_to_zero() {
|
||||||
|
let raw = KseiRawRow {
|
||||||
|
date: "27-Apr-2026".to_string(),
|
||||||
|
share_code: "AADI".to_string(),
|
||||||
|
issuer_name: "ADARO ANDALAN INDONESIA Tbk".to_string(),
|
||||||
|
investor_name: "ADARO STRATEGIC INVESTMENTS".to_string(),
|
||||||
|
investor_type: "CP".to_string(),
|
||||||
|
local_foreign: "D".to_string(),
|
||||||
|
nationality: "INDONESIA".to_string(),
|
||||||
|
domicile: String::new(),
|
||||||
|
holdings_scripless: String::new(),
|
||||||
|
holdings_scrip: String::new(),
|
||||||
|
total_holding_shares: "3.200.142.830".to_string(),
|
||||||
|
percentage: "66,18".to_string(),
|
||||||
|
};
|
||||||
|
|
||||||
|
let draft = normalize_ksei_row(&raw).expect("empty component shares should normalize");
|
||||||
|
assert_eq!(draft.holdings_scripless, 0);
|
||||||
|
assert_eq!(draft.holdings_scrip, 0);
|
||||||
|
assert_eq!(draft.total_shares, 3_200_142_830);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn test_normalize_ksei_row_still_requires_total_shares() {
|
||||||
|
let raw = KseiRawRow {
|
||||||
|
date: "27-Apr-2026".to_string(),
|
||||||
|
share_code: "AADI".to_string(),
|
||||||
|
issuer_name: "ADARO ANDALAN INDONESIA Tbk".to_string(),
|
||||||
|
investor_name: "ADARO STRATEGIC INVESTMENTS".to_string(),
|
||||||
|
investor_type: "CP".to_string(),
|
||||||
|
local_foreign: "D".to_string(),
|
||||||
|
nationality: "INDONESIA".to_string(),
|
||||||
|
domicile: String::new(),
|
||||||
|
holdings_scripless: String::new(),
|
||||||
|
holdings_scrip: String::new(),
|
||||||
|
total_holding_shares: String::new(),
|
||||||
|
percentage: "66,18".to_string(),
|
||||||
|
};
|
||||||
|
|
||||||
|
assert!(normalize_ksei_row(&raw).is_err());
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -223,16 +223,14 @@ fn detect_root_node(conn: &Connection, root: &str) -> Result<String, IdxError> {
|
||||||
fn query_all_edges(conn: &Connection) -> Result<Vec<GraphEdge>, IdxError> {
|
fn query_all_edges(conn: &Connection) -> Result<Vec<GraphEdge>, IdxError> {
|
||||||
let mut out = Vec::new();
|
let mut out = Vec::new();
|
||||||
|
|
||||||
if let Ok(release_sha) = conn
|
if let Ok(release_sha) = conn.query_row(
|
||||||
.query_row(
|
"SELECT sha256
|
||||||
"SELECT sha256
|
|
||||||
FROM ownership_releases
|
FROM ownership_releases
|
||||||
ORDER BY as_of_date DESC, imported_at DESC
|
ORDER BY as_of_date DESC, imported_at DESC
|
||||||
LIMIT 1",
|
LIMIT 1",
|
||||||
[],
|
[],
|
||||||
|row| row.get::<_, String>(0),
|
|row| row.get::<_, String>(0),
|
||||||
)
|
) {
|
||||||
{
|
|
||||||
let mut stmt = conn
|
let mut stmt = conn
|
||||||
.prepare(
|
.prepare(
|
||||||
"SELECT 'entity:' || k.entity_id,
|
"SELECT 'entity:' || k.entity_id,
|
||||||
|
|
@ -371,7 +369,7 @@ mod tests {
|
||||||
use chrono::NaiveDate;
|
use chrono::NaiveDate;
|
||||||
use rusqlite::Connection;
|
use rusqlite::Connection;
|
||||||
|
|
||||||
use crate::ownership::db::{ensure_schema, write_ksei_release, upsert_ticker};
|
use crate::ownership::db::{ensure_schema, upsert_ticker, write_ksei_release};
|
||||||
use crate::ownership::types::{KseiHolding, OwnershipRelease};
|
use crate::ownership::types::{KseiHolding, OwnershipRelease};
|
||||||
|
|
||||||
use super::query_ownership_graph;
|
use super::query_ownership_graph;
|
||||||
|
|
|
||||||
|
|
@ -355,23 +355,23 @@ fn parse_row_segments(texts: &[String]) -> KseiRawRow {
|
||||||
{
|
{
|
||||||
row.percentage = remaining.pop().unwrap_or_default();
|
row.percentage = remaining.pop().unwrap_or_default();
|
||||||
}
|
}
|
||||||
if remaining
|
let numeric_tail = pop_numeric_tail(&mut remaining);
|
||||||
.last()
|
match numeric_tail.as_slice() {
|
||||||
.is_some_and(|segment| is_id_number_like(segment))
|
[scripless, scrip, total] => {
|
||||||
{
|
row.holdings_scripless = scripless.clone();
|
||||||
row.total_holding_shares = remaining.pop().unwrap_or_default();
|
row.holdings_scrip = scrip.clone();
|
||||||
}
|
row.total_holding_shares = total.clone();
|
||||||
if remaining
|
}
|
||||||
.last()
|
// Some PDFs omit the zero-valued scrip column entirely.
|
||||||
.is_some_and(|segment| is_id_number_like(segment))
|
[scripless, total] => {
|
||||||
{
|
row.holdings_scripless = scripless.clone();
|
||||||
row.holdings_scrip = remaining.pop().unwrap_or_default();
|
row.holdings_scrip = "0".to_string();
|
||||||
}
|
row.total_holding_shares = total.clone();
|
||||||
if remaining
|
}
|
||||||
.last()
|
[total] => {
|
||||||
.is_some_and(|segment| is_id_number_like(segment))
|
row.total_holding_shares = total.clone();
|
||||||
{
|
}
|
||||||
row.holdings_scripless = remaining.pop().unwrap_or_default();
|
_ => {}
|
||||||
}
|
}
|
||||||
|
|
||||||
let mut geo_fields = Vec::new();
|
let mut geo_fields = Vec::new();
|
||||||
|
|
@ -439,6 +439,19 @@ fn is_share_code_like(value: &str) -> bool {
|
||||||
.all(|ch| ch.is_ascii_uppercase() || ch.is_ascii_digit())
|
.all(|ch| ch.is_ascii_uppercase() || ch.is_ascii_digit())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn pop_numeric_tail(remaining: &mut Vec<String>) -> Vec<String> {
|
||||||
|
let mut numeric_tail = Vec::new();
|
||||||
|
while numeric_tail.len() < 3
|
||||||
|
&& remaining
|
||||||
|
.last()
|
||||||
|
.is_some_and(|segment| is_id_number_like(segment))
|
||||||
|
{
|
||||||
|
numeric_tail.push(remaining.pop().unwrap_or_default());
|
||||||
|
}
|
||||||
|
numeric_tail.reverse();
|
||||||
|
numeric_tail
|
||||||
|
}
|
||||||
|
|
||||||
fn is_id_number_like(value: &str) -> bool {
|
fn is_id_number_like(value: &str) -> bool {
|
||||||
let trimmed = value.trim();
|
let trimmed = value.trim();
|
||||||
!trimmed.is_empty()
|
!trimmed.is_empty()
|
||||||
|
|
@ -554,8 +567,11 @@ fn is_percentage_like(s: &str) -> bool {
|
||||||
mod tests {
|
mod tests {
|
||||||
use std::path::Path;
|
use std::path::Path;
|
||||||
|
|
||||||
|
use crate::ownership::entities::normalize_ksei_row;
|
||||||
|
|
||||||
use super::{
|
use super::{
|
||||||
OwnershipPdfSchema, check_mutool, classify_stext_xml, parse_ksei_pdf, parse_stext_xml,
|
OwnershipPdfSchema, check_mutool, classify_stext_xml, parse_ksei_pdf, parse_row_segments,
|
||||||
|
parse_stext_xml,
|
||||||
};
|
};
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|
@ -608,6 +624,51 @@ mod tests {
|
||||||
assert_eq!(row.percentage, "41,10");
|
assert_eq!(row.percentage, "41,10");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn test_parse_row_segments_missing_scrip_defaults_to_zero() {
|
||||||
|
let texts = vec![
|
||||||
|
"27-Apr-2026 AADI".to_string(),
|
||||||
|
"ADARO ANDALAN INDONESIA Tbk".to_string(),
|
||||||
|
"ADARO STRATEGIC INVESTMENTS".to_string(),
|
||||||
|
"CP".to_string(),
|
||||||
|
"D".to_string(),
|
||||||
|
"INDONESIA".to_string(),
|
||||||
|
"3.200.142.830".to_string(),
|
||||||
|
"3.200.142.830".to_string(),
|
||||||
|
"66,18".to_string(),
|
||||||
|
];
|
||||||
|
|
||||||
|
let row = parse_row_segments(&texts);
|
||||||
|
assert_eq!(row.date, "27-Apr-2026");
|
||||||
|
assert_eq!(row.share_code, "AADI");
|
||||||
|
assert_eq!(row.holdings_scripless, "3.200.142.830");
|
||||||
|
assert_eq!(row.holdings_scrip, "0");
|
||||||
|
assert_eq!(row.total_holding_shares, "3.200.142.830");
|
||||||
|
assert_eq!(row.percentage, "66,18");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn test_parse_row_segments_missing_scrip_normalizes_without_error() {
|
||||||
|
let texts = vec![
|
||||||
|
"27-Apr-2026 AADI".to_string(),
|
||||||
|
"ADARO ANDALAN INDONESIA Tbk".to_string(),
|
||||||
|
"ADARO STRATEGIC INVESTMENTS".to_string(),
|
||||||
|
"CP".to_string(),
|
||||||
|
"D".to_string(),
|
||||||
|
"INDONESIA".to_string(),
|
||||||
|
"3.200.142.830".to_string(),
|
||||||
|
"3.200.142.830".to_string(),
|
||||||
|
"66,18".to_string(),
|
||||||
|
];
|
||||||
|
|
||||||
|
let row = parse_row_segments(&texts);
|
||||||
|
let draft = normalize_ksei_row(&row).expect("missing scrip column should normalize");
|
||||||
|
|
||||||
|
assert_eq!(draft.holdings_scripless, 3_200_142_830);
|
||||||
|
assert_eq!(draft.holdings_scrip, 0);
|
||||||
|
assert_eq!(draft.total_shares, 3_200_142_830);
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn classify_stext_xml_detects_supported_holder_register_schema() {
|
fn classify_stext_xml_detects_supported_holder_register_schema() {
|
||||||
let xml = include_str!("../../tests/fixtures/ksei_above1_stext_excerpt.xml");
|
let xml = include_str!("../../tests/fixtures/ksei_above1_stext_excerpt.xml");
|
||||||
|
|
|
||||||
21
tests/cli.rs
21
tests/cli.rs
|
|
@ -77,7 +77,8 @@ fn candidate_is_idx_binary(path: &Path) -> bool {
|
||||||
return false;
|
return false;
|
||||||
};
|
};
|
||||||
|
|
||||||
output.status.success() && String::from_utf8_lossy(&output.stdout).trim() == env!("CARGO_PKG_VERSION")
|
output.status.success()
|
||||||
|
&& String::from_utf8_lossy(&output.stdout).trim() == env!("CARGO_PKG_VERSION")
|
||||||
}
|
}
|
||||||
|
|
||||||
#[cfg(unix)]
|
#[cfg(unix)]
|
||||||
|
|
@ -1062,7 +1063,9 @@ fn ownership_releases_uses_xdg_data_home_default_db_path() {
|
||||||
.args(["ownership", "releases"])
|
.args(["ownership", "releases"])
|
||||||
.assert()
|
.assert()
|
||||||
.success()
|
.success()
|
||||||
.stdout(predicate::str::contains("No ownership releases imported yet."));
|
.stdout(predicate::str::contains(
|
||||||
|
"No ownership releases imported yet.",
|
||||||
|
));
|
||||||
|
|
||||||
assert!(
|
assert!(
|
||||||
data_home.join("idx").join("ownership.db").exists(),
|
data_home.join("idx").join("ownership.db").exists(),
|
||||||
|
|
@ -1584,7 +1587,15 @@ fn ownership_import_file_zip_archive_supports_releases_ticker_and_changes() {
|
||||||
.stdout(predicate::str::contains("KSEI AGGREGATE FOREIGN MF"));
|
.stdout(predicate::str::contains("KSEI AGGREGATE FOREIGN MF"));
|
||||||
|
|
||||||
let ticker_output = bin_with_root(&root)
|
let ticker_output = bin_with_root(&root)
|
||||||
.args(["-o", "json", "ownership", "ticker", "AADI", "--source", "ksei"])
|
.args([
|
||||||
|
"-o",
|
||||||
|
"json",
|
||||||
|
"ownership",
|
||||||
|
"ticker",
|
||||||
|
"AADI",
|
||||||
|
"--source",
|
||||||
|
"ksei",
|
||||||
|
])
|
||||||
.output()
|
.output()
|
||||||
.expect("ownership ticker json output");
|
.expect("ownership ticker json output");
|
||||||
assert!(ticker_output.status.success());
|
assert!(ticker_output.status.success());
|
||||||
|
|
@ -1860,7 +1871,9 @@ fn invalid_quote_ttl_env_returns_non_zero() {
|
||||||
.args(["version"])
|
.args(["version"])
|
||||||
.assert()
|
.assert()
|
||||||
.failure()
|
.failure()
|
||||||
.stderr(predicate::str::contains("invalid IDX_CACHE_QUOTE_TTL value"));
|
.stderr(predicate::str::contains(
|
||||||
|
"invalid IDX_CACHE_QUOTE_TTL value",
|
||||||
|
));
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue