Refactor KSEI PDF parsing for the live above-1 holder-register layout, add IDX announcement discovery plus browser-impersonated PDF fetches, and update the ownership docs to mark Batch 1 complete and scope Batch 2 around above1 hardening and unsupported legacy inputs.
- Change resolve_symbol return type from String to Result<String, IdxError>
- Add early validation for empty/whitespace-only ticker input
- Return Err(IdxError::InvalidInput) for empty tickers
- Add InvalidInput variant to IdxError enum
- Update all 15 callers in cli/stocks.rs to handle Result with ? propagation
- Update tests: empty/whitespace tickers now return error
Fixes#7
MSN Finance/Charts does not serve OHLCV data for XIDX stocks (returns 404).
Following the FP principle of not exposing capabilities a provider cannot fulfil:
- Remove impl HistoryProvider for MsnProvider entirely
- Remove fetch_charts from MsnClient
- Remove parse_chart_history, resample_history, trim_history_to_period from map.rs
- Remove MsnChart, ChartSeries, RawChart from raw_types.rs
- Remove parse_history_from_str, parse_close_only_history from parse.rs
- Decouple HistoryProvider from MarketDataProvider trait bound
- Add history_provider() factory: returns None for MSN, Some(Yahoo) for Yahoo
- CLI gates History/Technical on history_provider(), fails fast for MSN
- MSN mock returns Err(Unsupported); tests verify the behaviour explicitly
- financials: fix incomeStatement serde rename (was incomeStatements, API sends singular)
- financials: flatten nested sub-objects (income/revenue/expense/cash) in parse_statement_section
- insights: rewrite RawInsight to match actual API shape ({insights:[{insightName,insightStatement,category}]})
- insights: group insight items into highlights (non-risk) and risks by category
- screener: build Quote directly in parse_screener_results, skip stocks with no price
- profile: use short_name fallback when long_name is null, hide empty fields
- history: map Finance/Charts 404 to Unsupported with clear IDX-specific message
- tests: update MSN history/technical tests to use mock provider
Add MsnProvider implementing MarketDataProvider trait with quote,
fundamentals, and history support. Includes provider-aware config
(file/env/CLI), cache namespace isolation per provider, symbol ID
mapping via embedded TSV, OHLCV resampling, and comprehensive unit
+ integration tests with MSN fixture data.
Key changes:
- MsnProvider with quote, key-ratios, and chart endpoints
- ProviderKind enum (yahoo/msn) with config hierarchy support
- Provider-namespaced cache buckets to prevent cross-provider poisoning
- Provider-aware MockProvider loading correct fixtures per provider
- Integration tests for config round-trip and cache isolation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use curl_chrome131 (and fallback chain) instead of 'curl-impersonate --impersonate'
The curl-impersonate-chrome package ships per-version binaries, not a generic binary
- Add Unknown(serde_json::Value) catch-all variant to QuoteSummaryValue
Yahoo returns {} for empty fields and null/strings that broke deserialization
- Keep cookie jar flow: fc.yahoo.com (404 but writes A3 cookie) + getcrumb + quoteSummary
- Pass cookie header from jar to quoteSummary request via ureq
Live result: stocks fundamental/growth/valuation/risk/compare now working
BBCA: ROE 21.14% excellent, Net Margin 53.28% excellent, Growth mixed
Yahoo Finance blocks standard ureq/rustls via TLS fingerprinting (JA3/JA4).
Reverse-engineered from yfinance source: they use curl_cffi with Chrome impersonation.
Fix:
- Fetch cookie via curl-impersonate from fc.yahoo.com
- Fetch crumb via curl-impersonate from query1.finance.yahoo.com/v1/test/getcrumb
(correct URL: getcrumb, not getCrumb or csrfToken)
- Parse Netscape cookie jar format, send cookies as header to quoteSummary
- 401 retry: clear crumb + cookie jar, re-auth on next attempt
- Crumb validation: reject HTML, empty, rate-limit responses
- Add curl-impersonate-chrome to flake.nix devShell
