Newer KSEI above1 PDFs use spaces in column headers (SHARE CODE,
INVESTOR NAME, INVESTOR CLASSIFICATION) instead of the underscored
form (SHARE_CODE, INVESTOR_NAME, INVESTOR_TYPE) that the parser
schema markers expected. This caused valid holder-register PDFs to
be misclassified as LegacyAboveFivePercent (false positive on
REKENING TAMPUNGAN KSEI text in the data), blocking import.
Changes:
- Add normalize_stext_for_classification() that replaces spaces with
underscores inside text="..." attributes before schema marker
matching, so both old and new header layouts are recognized
- Add INVESTOR_CLASSIFICATION to HOLDER_REGISTER_SCHEMA_MARKERS and
HEADER_LABELS to match the renamed column
- Update ANNOUNCEMENT_WRAPPER and ABOVE_FIVE marker constants to use
underscores (matching the normalized text)
- Add test fixture and test case for the spaced-header layout
- Bump version to 0.2.3
Tested against the live June 2026 KSEI PDF (as_of 2026-05-29):
idx ownership import --url <latest-pdf-url>
-> Imported 7124 rows for 956 tickers (as of 2026-05-29)
The April 2026 IDX ownership PDF omits the zero-valued scrip column for
some holders, causing the parser to leave holdings_scripless/scrip as
empty strings. This broke normalize_ksei_row which called parse_id_number
on empty input.
Two-layer fix:
- Parser: pop_numeric_tail handles 2-column (missing scrip) rows by
defaulting scrip to "0"
- Normalizer: parse_id_number_or_zero backstop treats empty component
share fields as 0 while still requiring total_shares
Also includes AGENTS.md refresh, formatting cleanup (rustfmt), and
version bump to 0.2.2.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add a compact parser fixture excerpted from the real 2026-03-10 IDX above1 lamp1 mutool stext output and assert the first live row still parses as expected.
This keeps the post-merge ownership parser coverage grounded in the supported above1 layout instead of relying only on synthetic line fixtures.
Clear the smoke cache before each cache-group warm case so the following offline and stale-cache checks always start from a fresh baseline instead of earlier group state.
Document the runner behavior in the smoke guide to make the cache-group sequencing explicit.
Refactor KSEI PDF parsing for the live above-1 holder-register layout, add IDX announcement discovery plus browser-impersonated PDF fetches, and update the ownership docs to mark Batch 1 complete and scope Batch 2 around above1 hardening and unsupported legacy inputs.
Reject invalid values for known config keys in 'idx config set':
- general.provider: only yahoo|msn
- general.history_provider: only auto|yahoo|msn
- general.output: only table|json
- general.color: only true|false
- cache TTL fields: non-negative integers only
- Unknown keys: rejected with clear error listing valid keys
Issue #9: Previously, corrupted JSON in cache files would cause commands
to fail with a ParseError. Now:
- IO errors and parse errors are caught in read_entry()
- A warning is printed to stderr with the data_type/symbol
- The corrupted file is deleted
- Ok(None) is returned (treated as cache miss)
This applies to both get() and get_stale() since they share read_entry().
Added tests:
- corrupted_cache_entry_returns_none_and_deletes_file
- corrupted_cache_entry_get_stale_returns_none_and_deletes_file
Both verify the corrupted file is cleaned up after the graceful failure.
- Change resolve_symbol return type from String to Result<String, IdxError>
- Add early validation for empty/whitespace-only ticker input
- Return Err(IdxError::InvalidInput) for empty tickers
- Add InvalidInput variant to IdxError enum
- Update all 15 callers in cli/stocks.rs to handle Result with ? propagation
- Update tests: empty/whitespace tickers now return error
Fixes#7
