feat: support explicit MSN chart history

This commit is contained in:
Rasyidan Akbar F. 2026-04-15 13:48:27 +07:00
commit a2d15d2125
13 changed files with 280 additions and 68 deletions

View file

@ -49,7 +49,7 @@ The remaining work is architecture cleanup, a few correctness edge cases, and se
| Area | CLI | Status | Notes | | Area | CLI | Status | Notes |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| Quotes | `idx stocks quote` | Implemented | Cached, smoke-tested, and covered by integration tests | | Quotes | `idx stocks quote` | Implemented | Cached, smoke-tested, and covered by integration tests |
| History | `idx stocks history` | Partial | Works today via Yahoo/history-provider routing; explicit MSN history remains unsupported for IDX | | History | `idx stocks history` | Implemented with provider-specific limits | Yahoo remains the default/auto OHLCV source; explicit MSN history works for supported price-only chart windows |
| Technical | `idx stocks technical` | Implemented | Uses the cached history path | | Technical | `idx stocks technical` | Implemented | Uses the cached history path |
| Growth | `idx stocks growth` | Implemented | Shipped and exercised | | Growth | `idx stocks growth` | Implemented | Shipped and exercised |
| Valuation | `idx stocks valuation` | Implemented | Shipped and exercised | | Valuation | `idx stocks valuation` | Implemented | Shipped and exercised |
@ -63,11 +63,10 @@ The remaining work is architecture cleanup, a few correctness edge cases, and se
| Insights | `idx stocks insights` | Implemented | Summary/highlights/risks/`last_updated` mapping was corrected and tested | | Insights | `idx stocks insights` | Implemented | Summary/highlights/risks/`last_updated` mapping was corrected and tested |
| News | `idx stocks news` | Implemented | Fixture-backed CLI coverage exists | | News | `idx stocks news` | Implemented | Fixture-backed CLI coverage exists |
| Screener | `idx stocks screen` | Implemented with gaps | Validation landed; expression/preset workflow is still future work | | Screener | `idx stocks screen` | Implemented with gaps | Validation landed; expression/preset workflow is still future work |
| MSN charts | `idx stocks history --history-provider msn` | Missing | Explicit MSN history still returns unsupported for IDX | | MSN charts | `idx stocks history --history-provider msn` | Implemented with limits | Supports `--period 1mo|3mo|1y --interval 1d`; MSN provides price-only chart series, so OHLC is synthesized and volume is `0` |
| KSEI ownership import/query | `idx ownership import --file`, `idx ownership import --url`, `idx ownership releases`, `idx ownership ticker` | Implemented | Local PDF import and SQLite-backed query flow are verified against the March 2026 KSEI release; remote IDX import now works for the discovered `above 1%` `lamp1` BEI attachment, and legacy `above 5%` / `investor-type` BEI report families are rejected explicitly | | KSEI ownership import/query | `idx ownership import --file`, `idx ownership import --url`, `idx ownership releases`, `idx ownership ticker` | Implemented | Local PDF import and SQLite-backed query flow are verified against the March 2026 KSEI release; remote IDX import now works for the discovered `above 1%` `lamp1` BEI attachment, and legacy `above 5%` / `investor-type` BEI report families are rejected explicitly |
| KSEI archive fallback import | `idx ownership import --file <.zip|.txt>` | Implemented as fallback | Local archive ZIP/TXT ingest maps investor-type/locality buckets into synthetic aggregate holders for validation/backstop use, not the primary ingest surface | | KSEI archive fallback import | `idx ownership import --file <.zip|.txt>` | Implemented as fallback | Local archive ZIP/TXT ingest maps investor-type/locality buckets into synthetic aggregate holders for validation/backstop use, not the primary ingest surface |
| Ownership snapshot sync | `idx ownership sync` | Implemented | Manifest-driven SQLite snapshot install with checksum validation, conservative replacement/no-op rules, and publisher helper script | | Ownership snapshot sync | `idx ownership sync` | Implemented | Manifest-driven SQLite snapshot install with checksum validation, conservative replacement/no-op rules, and publisher helper script |
| Bing ownership CLI | `idx ownership import --fetch-bing` | Not implemented | Client groundwork exists, CLI import path is still deferred |
--- ---
@ -166,18 +165,10 @@ Done when:
Priority order: Priority order:
1. MSN Charts / `Finance/Charts` 1. Richer financial statements
- Reuse the existing `idx stocks history` command.
- Decide how to handle price-only timeframes safely.
2. Bing ownership CLI integration
- Reuse the existing client groundwork in `src/api/msn/bing.rs`.
- Define the import shape and output contract for `idx ownership import --fetch-bing`.
3. Richer financial statements
- Decide whether to stay with the current single-period model or add multi-period fetch support. - Decide whether to stay with the current single-period model or add multi-period fetch support.
4. New user-facing surfaces from `TODO.md` 2. New user-facing surfaces from `TODO.md`
- `market summary` - `market summary`
- `market movers` - `market movers`
- `market sectors` - `market sectors`
@ -215,7 +206,5 @@ MSN API key (public, embedded in MSN Money website):
Base URLs: Base URLs:
- `https://assets.msn.com/service/` - core market data (Quotes, Charts, Equities, Earnings, Sentiment, Screener) - `https://assets.msn.com/service/` - core market data (Quotes, Charts, Equities, Earnings, Sentiment, Screener)
- `https://api.msn.com/msn/v0/pages/finance/` - extended data (key ratios, insights, news feed) - `https://api.msn.com/msn/v0/pages/finance/` - extended data (key ratios, insights, news feed)
- `https://services.bingapis.com/contentservices-finance.hedgefunddataprovider/api/v1/` - Bing ownership data
Keep this appendix for endpoint discovery and future work. Keep this appendix for endpoint discovery and future work.
Use the sections above as the actual implementation plan. Use the sections above as the actual implementation plan.

View file

@ -129,8 +129,7 @@
- [x] Decide whether `screen` stays under `stocks` long term or graduates into a richer dedicated surface later - [x] Decide whether `screen` stays under `stocks` long term or graduates into a richer dedicated surface later
### P2 — Deferred but real work ### P2 — Deferred but real work
- [ ] Add MSN chart/history support through `stocks history --history-provider msn` - [x] Add MSN chart/history support through `stocks history --history-provider msn` for supported price-only daily chart windows (`1mo`, `3mo`, `1y`)
- [ ] Define and implement `ownership import --fetch-bing`
- [ ] Decide whether richer financial statements should stay single-period or grow into multi-period fetch support - [ ] Decide whether richer financial statements should stay single-period or grow into multi-period fetch support
## 🚀 Publish Readiness (2026-04-03) ## 🚀 Publish Readiness (2026-04-03)
@ -182,7 +181,7 @@
- [x] Verification on `2026-04-02`: `nix develop --command cargo build`, `nix develop --command cargo clippy -- -D warnings`, `nix develop --command cargo test`, `scripts/live-smoke.sh --mode mock`, and `scripts/live-smoke.sh --group live-table --group live-json --group routing --group cache --group errors` all passed (`tmp/live-smoke/20260402-122215`, `tmp/live-smoke/20260402-122218`) - [x] Verification on `2026-04-02`: `nix develop --command cargo build`, `nix develop --command cargo clippy -- -D warnings`, `nix develop --command cargo test`, `scripts/live-smoke.sh --mode mock`, and `scripts/live-smoke.sh --group live-table --group live-json --group routing --group cache --group errors` all passed (`tmp/live-smoke/20260402-122215`, `tmp/live-smoke/20260402-122218`)
- [x] Live smoke passed for shipped `stocks` commands: `quote`, `history`, `technical`, `growth`, `valuation`, `risk`, `fundamental`, `compare`, `profile`, `financials`, `earnings`, `sentiment`, `insights`, `news`, `screen` - [x] Live smoke passed for shipped `stocks` commands: `quote`, `history`, `technical`, `growth`, `valuation`, `risk`, `fundamental`, `compare`, `profile`, `financials`, `earnings`, `sentiment`, `insights`, `news`, `screen`
- [x] Yahoo routing verified for live `quote` and `history` - [x] Yahoo routing verified for live `quote` and `history`
- [x] `stocks history --history-provider msn` correctly fails for IDX as unsupported - [x] `stocks history --history-provider msn` now works for supported MSN price-only chart windows; Yahoo remains the auto history source for full OHLCV
- [x] `ownership releases` works with a writable `ownership.db_path` and an empty DB - [x] `ownership releases` works with a writable `ownership.db_path` and an empty DB
- [x] Regression coverage now verifies offline/cache parity for MSN-only commands (`stocks profile BBCA`) - [x] Regression coverage now verifies offline/cache parity for MSN-only commands (`stocks profile BBCA`)
- [x] `--offline --no-cache` now fails fast as an invalid flag combination instead of serving cache - [x] `--offline --no-cache` now fails fast as an invalid flag combination instead of serving cache
@ -208,7 +207,6 @@
- [x] MSN key-ratios parsing now tolerates stringified non-finite numeric sentinels (`Infinity`, `-Infinity`, `NaN`) as missing data instead of failing the entire fundamentals payload - [x] MSN key-ratios parsing now tolerates stringified non-finite numeric sentinels (`Infinity`, `-Infinity`, `NaN`) as missing data instead of failing the entire fundamentals payload
- [x] Live direct CLI repros on `2026-04-13` confirmed the original failure class on `BUMI`, `ADRO`, and `AIMS`; an opt-in `scripts/live-smoke.sh --group live-nonfinite` group now covers those real ticker paths - [x] Live direct CLI repros on `2026-04-13` confirmed the original failure class on `BUMI`, `ADRO`, and `AIMS`; an opt-in `scripts/live-smoke.sh --group live-nonfinite` group now covers those real ticker paths
- [x] Added `scripts/audit-msn-fundamentals.sh` for a full CLI valuation sweep across the IDX MSN symbol map; keep it as a provider-health audit, not part of the default smoke baseline - [x] Added `scripts/audit-msn-fundamentals.sh` for a full CLI valuation sweep across the IDX MSN symbol map; keep it as a provider-health audit, not part of the default smoke baseline
- [ ] `ownership import --fetch-bing` is still deferred and returns unsupported
- [x] Real KSEI PDF import from local file now works again: `ownership import --file /Users/rasyidanakbar/Downloads/ksei_raw_data.pdf` imported `7261` rows for `955` tickers on `2026-03-28`, replacing the previous `6`-row/`1`-ticker failure mode - [x] Real KSEI PDF import from local file now works again: `ownership import --file /Users/rasyidanakbar/Downloads/ksei_raw_data.pdf` imported `7261` rows for `955` tickers on `2026-03-28`, replacing the previous `6`-row/`1`-ticker failure mode
- [x] KSEI parser no longer depends on the old hardcoded column bounds fixture layout; it now reconstructs rows from `mutool` line output and handles the live `DATE + SHARE_CODE` merged segment plus `D`/`A` locality markers - [x] KSEI parser no longer depends on the old hardcoded column bounds fixture layout; it now reconstructs rows from `mutool` line output and handles the live `DATE + SHARE_CODE` merged segment plus `D`/`A` locality markers
- [x] A real IDX-hosted March 2026 ownership PDF URL was verified on `2026-03-29`, but only through `curl-impersonate` inside `nix develop`; plain `curl` still returns `403` from Cloudflare for the same asset - [x] A real IDX-hosted March 2026 ownership PDF URL was verified on `2026-03-29`, but only through `curl-impersonate` inside `nix develop`; plain `curl` still returns `403` from Cloudflare for the same asset

View file

@ -31,14 +31,17 @@
## Provider Architecture ## Provider Architecture
### Dual Provider Model ### Dual Provider Model
- **MSN Finance** — default provider. Rich data: quotes, fundamentals, profile, earnings, financials, sentiment, insights, news, screener. No history for IDX stocks. - **MSN Finance** — default provider. Rich data: quotes, fundamentals, profile, earnings, financials, sentiment, insights, news, screener, and explicit price-only chart history for supported IDX windows.
- **Yahoo Finance**history fallback. Reliable OHLCV data via `/v8/finance/chart/`. - **Yahoo Finance**default/auto history source. Reliable OHLCV data via `/v8/finance/chart/`.
### Hybrid History Strategy ### Hybrid History Strategy
When `history_provider = auto` (default): When `history_provider = auto` (default):
1. Check if current provider supports `HistoryProvider` trait 1. Use Yahoo for history because it provides full OHLCV candles.
2. MSN doesn't → transparently fallback to Yahoo 2. Keep logging when provider selection falls back from MSN to Yahoo.
3. Log info message: `"history provider fallback active (msn -> yahoo)"` 3. Allow explicit `--history-provider msn` for supported MSN chart windows (`1mo`, `3mo`, `1y` with `1d` interval).
MSN Charts are price-only for IDX. The CLI normalizes them into `Ohlc` rows by
using the chart price as open/high/low/close and `0` volume.
### Capability Gating ### Capability Gating
``` ```

View file

@ -35,7 +35,7 @@ scripts/audit-msn-fundamentals.sh --tickers BUMI,ADRO,AIMS
- `live-json`: all shipped `stocks` commands in live JSON mode - `live-json`: all shipped `stocks` commands in live JSON mode
- `mock`: all shipped `stocks` commands against the mock provider in both table and JSON mode - `mock`: all shipped `stocks` commands against the mock provider in both table and JSON mode
- `cache`: cache warm, `--offline`, and stale-cache fallback checks for quote, technical, and MSN `profile` - `cache`: cache warm, `--offline`, and stale-cache fallback checks for quote, technical, and MSN `profile`
- `routing`: Yahoo/MSN provider routing plus explicit MSN history unsupported behavior - `routing`: Yahoo/MSN provider routing plus explicit MSN history behavior
- `errors`: JSON error contract and invalid flag/input checks - `errors`: JSON error contract and invalid flag/input checks
- `live-nonfinite`: opt-in live MSN fundamentals checks for known non-finite ticker payloads (`BUMI`, `ADRO`, `AIMS`) - `live-nonfinite`: opt-in live MSN fundamentals checks for known non-finite ticker payloads (`BUMI`, `ADRO`, `AIMS`)
- `ownership`: safe ownership smoke checks that do not require imported ownership data - `ownership`: safe ownership smoke checks that do not require imported ownership data

View file

@ -249,7 +249,7 @@ register_cases() {
add_case "routing" "yahoo-quote" "0" "$yahoo_mock_env" "stocks quote $live_symbol" "" add_case "routing" "yahoo-quote" "0" "$yahoo_mock_env" "stocks quote $live_symbol" ""
add_case "routing" "auto-history-fallback" "0" "$mock_env" "$history_args" "" add_case "routing" "auto-history-fallback" "0" "$mock_env" "$history_args" ""
add_case "routing" "auto-technical-fallback" "0" "$mock_env" "stocks technical $live_symbol" "" add_case "routing" "auto-technical-fallback" "0" "$mock_env" "stocks technical $live_symbol" ""
add_case "routing" "explicit-msn-history-unsupported" "1" "$mock_env" "stocks history $live_symbol --period 3mo --history-provider msn" "MSN does not provide price history" add_case "routing" "explicit-msn-history" "0" "$mock_env" "stocks history $live_symbol --period 3mo --history-provider msn" "History for"
add_case "errors" "invalid-provider-json" "1" "IDX_PROVIDER=bogus" "-o json version" "\"error\": true" add_case "errors" "invalid-provider-json" "1" "IDX_PROVIDER=bogus" "-o json version" "\"error\": true"
add_case "errors" "profile-provider-gate-json" "1" "IDX_PROVIDER=yahoo" "-o json stocks profile $live_symbol" "requires --provider msn" add_case "errors" "profile-provider-gate-json" "1" "IDX_PROVIDER=yahoo" "-o json stocks profile $live_symbol" "requires --provider msn"

View file

@ -215,8 +215,8 @@ fn msn_capability_error(subject: &str) -> IdxError {
/// Resolves a history provider based on the selected market data provider and /// Resolves a history provider based on the selected market data provider and
/// history provider strategy. /// history provider strategy.
/// ///
/// `history_mode=auto` means: use the selected provider when it supports history, /// `history_mode=auto` keeps using Yahoo for IDX history because Yahoo provides
/// otherwise transparently fallback to Yahoo. /// full OHLCV candles. Explicit `msn` opts into MSN's price-only chart feed.
pub fn history_provider( pub fn history_provider(
provider: ProviderKind, provider: ProviderKind,
history_mode: HistoryProviderKind, history_mode: HistoryProviderKind,
@ -232,12 +232,6 @@ pub fn history_provider(
}; };
if std::env::var("IDX_USE_MOCK_PROVIDER").is_ok() { if std::env::var("IDX_USE_MOCK_PROVIDER").is_ok() {
if matches!(resolved, ProviderKind::Msn) {
return Err(IdxError::Unsupported(
"MSN does not provide price history for IDX stocks. Use --history-provider yahoo or auto."
.into(),
));
}
return Ok(( return Ok((
resolved, resolved,
Box::new(MockProvider::from_fixtures_with_history_verbose( Box::new(MockProvider::from_fixtures_with_history_verbose(
@ -248,10 +242,7 @@ pub fn history_provider(
match resolved { match resolved {
ProviderKind::Yahoo => Ok((resolved, Box::new(yahoo::YahooProvider::new(verbose)))), ProviderKind::Yahoo => Ok((resolved, Box::new(yahoo::YahooProvider::new(verbose)))),
ProviderKind::Msn => Err(IdxError::Unsupported( ProviderKind::Msn => Ok((resolved, Box::new(msn::MsnProvider::new(verbose)))),
"MSN does not provide price history for IDX stocks. Use --history-provider yahoo or auto."
.into(),
)),
} }
} }
@ -317,10 +308,10 @@ impl MockProvider {
.map_err(|e| IdxError::ParseError(e.to_string())); .map_err(|e| IdxError::ParseError(e.to_string()));
let fundamentals = msn::parse_fundamentals_from_str(&fundamentals_raw, Some(&quote_raw)) let fundamentals = msn::parse_fundamentals_from_str(&fundamentals_raw, Some(&quote_raw))
.map_err(|e| IdxError::ParseError(e.to_string())); .map_err(|e| IdxError::ParseError(e.to_string()));
// MSN Finance/Charts returns 404 for IDX (XIDX) — history not supported let history_raw = std::fs::read_to_string("tests/fixtures/msn_chart_bbca_3m.json")
let history = Err(IdxError::Unsupported( .unwrap_or_else(|_| "[]".to_string());
"MSN does not provide price history for IDX stocks. Use --history-provider yahoo or auto.".into(), let history = msn::parse_history_from_str("BBCA.JK", &history_raw)
)); .map_err(|e| IdxError::ParseError(e.to_string()));
Self { Self {
quote, quote,

View file

@ -6,10 +6,12 @@ use serde::de::DeserializeOwned;
use crate::error::IdxError; use crate::error::IdxError;
use super::raw_types::{ use super::raw_types::{
KeyRatios, MsnQuote, RawEarningsResponse, RawEquity, RawFinancialStatement, RawInsight, KeyRatios, MsnQuote, RawChartResponse, RawEarningsResponse, RawEquity, RawFinancialStatement,
RawNewsFeed, RawScreenerResponse, RawSentiment, ScreenerFilter, ScreenerOrder, ScreenerRequest, RawInsight, RawNewsFeed, RawScreenerResponse, RawSentiment, ScreenerFilter, ScreenerOrder,
ScreenerRequest,
}; };
use super::symbols::resolve_msn_id; use super::symbols::resolve_msn_id;
use crate::api::types::{Interval, Period};
const USER_AGENT: &str = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36"; const USER_AGENT: &str = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36";
const MSN_ASSETS_BASE_URL: &str = "https://assets.msn.com/service/"; const MSN_ASSETS_BASE_URL: &str = "https://assets.msn.com/service/";
@ -58,6 +60,7 @@ impl MsnClient {
"insights" => include_str!("../../../tests/fixtures/msn_insights_bbca.json"), "insights" => include_str!("../../../tests/fixtures/msn_insights_bbca.json"),
"news" => include_str!("../../../tests/fixtures/msn_news_bbca.json"), "news" => include_str!("../../../tests/fixtures/msn_news_bbca.json"),
"screener" => include_str!("../../../tests/fixtures/msn_screener_id_topperfs.json"), "screener" => include_str!("../../../tests/fixtures/msn_screener_id_topperfs.json"),
"chart" => include_str!("../../../tests/fixtures/msn_chart_bbca_3m.json"),
_ => return None, _ => return None,
}) })
} }
@ -296,4 +299,68 @@ impl MsnClient {
self.post_json(&url, &req, "SCREENER", "screener") self.post_json(&url, &req, "SCREENER", "screener")
} }
pub(super) fn fetch_chart(
&self,
symbol: &str,
period: &Period,
interval: &Interval,
) -> Result<Vec<RawChartResponse>, IdxError> {
let id =
resolve_msn_id(symbol).ok_or_else(|| IdxError::SymbolNotFound(symbol.to_string()))?;
let chart_type = msn_chart_type(period, interval)?;
let url = format!(
"{MSN_ASSETS_BASE_URL}Finance/Charts?apikey={MSN_API_KEY}&cm=id-id&ids={id}&type={chart_type}&wrapodata=false"
);
self.get_json(&url, symbol, "chart")
}
}
fn msn_chart_type(period: &Period, interval: &Interval) -> Result<&'static str, IdxError> {
if !matches!(interval, Interval::Day) {
return Err(IdxError::Unsupported(
"MSN charts currently support only --interval 1d for IDX history".into(),
));
}
match period {
Period::OneMonth => Ok("1M"),
Period::ThreeMonths => Ok("3M"),
Period::OneYear => Ok("1Y"),
_ => Err(IdxError::Unsupported(
"MSN charts currently support --period 1mo, 3mo, or 1y with --interval 1d".into(),
)),
}
}
#[cfg(test)]
mod tests {
use super::msn_chart_type;
use crate::api::types::{Interval, Period};
use crate::error::IdxError;
#[test]
fn maps_supported_msn_chart_types() {
assert_eq!(
msn_chart_type(&Period::OneMonth, &Interval::Day).unwrap(),
"1M"
);
assert_eq!(
msn_chart_type(&Period::ThreeMonths, &Interval::Day).unwrap(),
"3M"
);
assert_eq!(
msn_chart_type(&Period::OneYear, &Interval::Day).unwrap(),
"1Y"
);
}
#[test]
fn rejects_unsupported_msn_chart_types() {
let err = msn_chart_type(&Period::ThreeMonths, &Interval::Week).unwrap_err();
assert!(matches!(err, IdxError::Unsupported(_)));
let err = msn_chart_type(&Period::SixMonths, &Interval::Day).unwrap_err();
assert!(matches!(err, IdxError::Unsupported(_)));
}
} }

View file

@ -1,14 +1,16 @@
use super::raw_types::{ use super::raw_types::{
IndustryMetric, KeyRatios, MsnQuote, RawEarningsData, RawEarningsResponse, RawEquity, IndustryMetric, KeyRatios, MsnQuote, RawChartResponse, RawChartSeries, RawEarningsData,
RawFinancialStatement, RawInsight, RawInsightItem, RawLocalizedAttribute, RawNewsFeed, RawEarningsResponse, RawEquity, RawFinancialStatement, RawInsight, RawInsightItem,
RawScreenerResponse, RawSentiment, RawStatementSection, RawLocalizedAttribute, RawNewsFeed, RawScreenerResponse, RawSentiment, RawStatementSection,
}; };
use super::symbols::{normalized_symbol, ticker_from_symbol}; use super::symbols::{normalized_symbol, ticker_from_symbol};
use crate::api::types::{ use crate::api::types::{
CompanyProfile, EarningsData, EarningsReport, FinancialStatements, Fundamentals, InsightData, CompanyProfile, EarningsData, EarningsReport, FinancialStatements, Fundamentals, InsightData,
InstrumentInfo, NewsItem, Officer, Quote, SentimentData, SentimentPeriod, StatementSection, InstrumentInfo, NewsItem, Officer, Ohlc, Quote, SentimentData, SentimentPeriod,
StatementSection,
}; };
use crate::error::IdxError; use crate::error::IdxError;
use chrono::DateTime;
use std::collections::HashMap; use std::collections::HashMap;
pub(super) fn parse_quote(symbol: &str, quotes: &[MsnQuote]) -> Result<Quote, IdxError> { pub(super) fn parse_quote(symbol: &str, quotes: &[MsnQuote]) -> Result<Quote, IdxError> {
@ -628,6 +630,74 @@ pub(super) fn parse_screener_results(raw: &RawScreenerResponse) -> Result<Vec<Qu
Ok(results) Ok(results)
} }
pub(super) fn parse_history(
symbol: &str,
charts: &[RawChartResponse],
) -> Result<Vec<Ohlc>, IdxError> {
let chart = charts.first().ok_or(IdxError::ProviderUnavailable)?;
let series = chart.series.as_ref().ok_or(IdxError::ProviderUnavailable)?;
let mut out = Vec::new();
for (idx, raw_ts) in series.time_stamps.iter().enumerate() {
let Some(close_raw) = series.prices.get(idx).copied().flatten() else {
continue;
};
if !close_raw.is_finite() {
continue;
}
let timestamp = DateTime::parse_from_rfc3339(raw_ts)
.map_err(|e| IdxError::ParseError(format!("msn chart timestamp '{raw_ts}': {e}")))?;
let close = round_price(close_raw);
let open = series_price_at(&series.open_prices, idx).unwrap_or(close);
let high = series_price_at(&series.prices_high, idx).unwrap_or(close);
let low = series_price_at(&series.prices_low, idx).unwrap_or(close);
let volume = series_volume_at(series, idx);
out.push(Ohlc {
date: timestamp.date_naive(),
open,
high,
low,
close,
volume,
});
}
if out.is_empty() {
return Err(IdxError::ProviderUnavailable);
}
if let Some(raw_symbol) = chart.symbol.as_deref()
&& let (Some(expected), Some(actual)) =
(ticker_from_symbol(symbol), ticker_from_symbol(raw_symbol))
&& expected != actual
{
return Err(IdxError::SymbolNotFound(symbol.to_string()));
}
Ok(out)
}
fn series_price_at(values: &[Option<f64>], idx: usize) -> Option<i64> {
values
.get(idx)
.copied()
.flatten()
.filter(|value| value.is_finite())
.map(round_price)
}
fn series_volume_at(series: &RawChartSeries, idx: usize) -> u64 {
series
.volumes
.get(idx)
.copied()
.flatten()
.filter(|value| value.is_finite() && !value.is_sign_negative())
.map(|value| value.round() as u64)
.unwrap_or(0)
}
fn parse_statement_section(section: &RawStatementSection) -> StatementSection { fn parse_statement_section(section: &RawStatementSection) -> StatementSection {
// MSN financial statement values are nested one level deep inside sub-objects // MSN financial statement values are nested one level deep inside sub-objects
// (e.g., incomeStatement.income.{lineItems}, incomeStatement.revenue.{lineItems}) // (e.g., incomeStatement.income.{lineItems}, incomeStatement.revenue.{lineItems})
@ -705,9 +775,9 @@ fn collect_earnings(
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::{ use super::{
KeyRatios, RawFinancialStatement, RawNewsFeed, RawScreenerResponse, RawSentiment, KeyRatios, RawChartResponse, RawFinancialStatement, RawNewsFeed, RawScreenerResponse,
parse_financial_statements, parse_fundamentals, parse_news, parse_screener_results, RawSentiment, parse_financial_statements, parse_fundamentals, parse_history, parse_news,
parse_sentiment, parse_screener_results, parse_sentiment,
}; };
use crate::error::IdxError; use crate::error::IdxError;
@ -861,4 +931,21 @@ mod tests {
"unsupported: company fundamentals unavailable from MSN; industry fallback is disabled" "unsupported: company fundamentals unavailable from MSN; industry fallback is disabled"
); );
} }
#[test]
fn parses_msn_chart_price_only_fixture_as_synthetic_ohlc() {
let raw: Vec<RawChartResponse> = serde_json::from_str(include_str!(
"../../../tests/fixtures/msn_chart_bbca_3m.json"
))
.expect("chart fixture should deserialize");
let history = parse_history("BBCA.JK", &raw).expect("chart history should parse");
assert_eq!(history.len(), 3);
assert_eq!(history[0].date.to_string(), "2026-01-13");
assert_eq!(history[0].open, 8000);
assert_eq!(history[0].high, 8000);
assert_eq!(history[0].low, 8000);
assert_eq!(history[0].close, 8000);
assert_eq!(history[0].volume, 0);
}
} }

View file

@ -7,22 +7,22 @@ mod raw_types;
mod symbols; mod symbols;
use crate::api::types::{ use crate::api::types::{
CompanyProfile, EarningsReport, FinancialStatements, Fundamentals, InsightData, NewsItem, Bar, CompanyProfile, EarningsReport, FinancialStatements, Fundamentals, InsightData, Interval,
Quote, SentimentData, NewsItem, Period, Quote, SentimentData,
}; };
use crate::api::{ use crate::api::{
EarningsProvider, FinancialsProvider, FundamentalsProvider, InsightsProvider, NewsProvider, EarningsProvider, FinancialsProvider, FundamentalsProvider, HistoryProvider, InsightsProvider,
ProfileProvider, QuoteProvider, ScreenerProvider, SentimentProvider, NewsProvider, ProfileProvider, QuoteProvider, ScreenerProvider, SentimentProvider,
}; };
use crate::error::IdxError; use crate::error::IdxError;
use client::MsnClient; use client::MsnClient;
use map::{ use map::{
parse_earnings, parse_financial_statements, parse_fundamentals, parse_insights, parse_news, parse_earnings, parse_financial_statements, parse_fundamentals, parse_history, parse_insights,
parse_profile, parse_quote, parse_screener_results, parse_sentiment, parse_news, parse_profile, parse_quote, parse_screener_results, parse_sentiment,
}; };
pub(crate) use parse::{parse_fundamentals_from_str, parse_quote_from_str}; pub(crate) use parse::{parse_fundamentals_from_str, parse_history_from_str, parse_quote_from_str};
pub struct MsnProvider { pub struct MsnProvider {
client: MsnClient, client: MsnClient,
@ -51,6 +51,18 @@ impl FundamentalsProvider for MsnProvider {
} }
} }
impl HistoryProvider for MsnProvider {
fn history(
&self,
symbol: &str,
period: &Period,
interval: &Interval,
) -> Result<Vec<Bar>, IdxError> {
let raw = self.client.fetch_chart(symbol, period, interval)?;
parse_history(symbol, &raw)
}
}
impl ProfileProvider for MsnProvider { impl ProfileProvider for MsnProvider {
fn profile(&self, symbol: &str) -> Result<CompanyProfile, IdxError> { fn profile(&self, symbol: &str) -> Result<CompanyProfile, IdxError> {
let raw = self.client.fetch_equities(symbol)?; let raw = self.client.fetch_equities(symbol)?;

View file

@ -1,6 +1,6 @@
use super::map::{parse_fundamentals, parse_quote}; use super::map::{parse_fundamentals, parse_history, parse_quote};
use super::raw_types::{KeyRatios, MsnQuote}; use super::raw_types::{KeyRatios, MsnQuote, RawChartResponse};
use crate::api::types::{Fundamentals, Quote}; use crate::api::types::{Fundamentals, Ohlc, Quote};
use crate::error::IdxError; use crate::error::IdxError;
#[cfg_attr(not(test), allow(dead_code))] #[cfg_attr(not(test), allow(dead_code))]
@ -25,11 +25,18 @@ pub(crate) fn parse_fundamentals_from_str(
parse_fundamentals(&ratios, quote.as_ref()) parse_fundamentals(&ratios, quote.as_ref())
} }
#[cfg_attr(not(test), allow(dead_code))]
pub(crate) fn parse_history_from_str(symbol: &str, raw: &str) -> Result<Vec<Ohlc>, IdxError> {
let charts: Vec<RawChartResponse> =
serde_json::from_str(raw).map_err(|e| IdxError::ParseError(e.to_string()))?;
parse_history(symbol, &charts)
}
#[cfg_attr(not(test), allow(dead_code))] #[cfg_attr(not(test), allow(dead_code))]
#[allow(dead_code)] #[allow(dead_code)]
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::{parse_fundamentals_from_str, parse_quote_from_str}; use super::{parse_fundamentals_from_str, parse_history_from_str, parse_quote_from_str};
fn minimal_quote_raw() -> &'static str { fn minimal_quote_raw() -> &'static str {
r#"[{"symbol":"BBCA","marketCap":1215200000000000}]"# r#"[{"symbol":"BBCA","marketCap":1215200000000000}]"#
@ -64,6 +71,17 @@ mod tests {
assert_eq!(fundamentals.market_cap, Some(1_215_200_000_000_000)); assert_eq!(fundamentals.market_cap, Some(1_215_200_000_000_000));
} }
#[test]
fn parses_history_fixture_json() {
let raw = std::fs::read_to_string("tests/fixtures/msn_chart_bbca_3m.json")
.expect("chart fixture exists");
let history = parse_history_from_str("BBCA.JK", &raw).expect("chart fixture parsed");
assert_eq!(history.len(), 3);
assert_eq!(history[0].date.to_string(), "2026-01-13");
assert_eq!(history[0].close, 8000);
}
#[test] #[test]
fn parses_fundamentals_with_infinity_string_as_missing_data() { fn parses_fundamentals_with_infinity_string_as_missing_data() {
let raw = r#"[ let raw = r#"[

View file

@ -319,6 +319,37 @@ pub(super) struct RawScreenerResponse {
pub(super) quote: Option<Vec<MsnQuote>>, pub(super) quote: Option<Vec<MsnQuote>>,
} }
#[allow(dead_code)]
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct RawChartResponse {
#[serde(rename = "_p")]
pub(super) id: Option<String>,
pub(super) chart_type: Option<String>,
pub(super) symbol: Option<String>,
pub(super) series: Option<RawChartSeries>,
}
#[allow(dead_code)]
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct RawChartSeries {
#[serde(default)]
pub(super) time_stamps: Vec<String>,
#[serde(default)]
pub(super) prices: Vec<Option<f64>>,
#[serde(default)]
pub(super) open_prices: Vec<Option<f64>>,
#[serde(default)]
pub(super) prices_high: Vec<Option<f64>>,
#[serde(default)]
pub(super) prices_low: Vec<Option<f64>>,
#[serde(default)]
pub(super) volumes: Vec<Option<f64>>,
pub(super) start_time: Option<String>,
pub(super) end_time: Option<String>,
}
fn de_opt_f64_lenient<'de, D>(deserializer: D) -> Result<Option<f64>, D::Error> fn de_opt_f64_lenient<'de, D>(deserializer: D) -> Result<Option<f64>, D::Error>
where where
D: Deserializer<'de>, D: Deserializer<'de>,

View file

@ -520,8 +520,8 @@ fn msn_technical_auto_falls_back_to_yahoo() {
} }
#[test] #[test]
fn explicit_msn_history_provider_returns_unsupported() { fn explicit_msn_history_provider_uses_msn_chart_fixture() {
test_bin("msn-history-explicit-unsupported") test_bin("msn-history-explicit")
.env("IDX_PROVIDER", "msn") .env("IDX_PROVIDER", "msn")
.env("IDX_USE_MOCK_PROVIDER", "1") .env("IDX_USE_MOCK_PROVIDER", "1")
.args([ .args([
@ -534,10 +534,9 @@ fn explicit_msn_history_provider_returns_unsupported() {
"msn", "msn",
]) ])
.assert() .assert()
.failure() .success()
.stderr(predicate::str::contains( .stdout(predicate::str::contains("History for BBCA.JK"))
"MSN does not provide price history", .stdout(predicate::str::contains("8,000"));
));
} }
#[test] #[test]

17
tests/fixtures/msn_chart_bbca_3m.json vendored Normal file
View file

@ -0,0 +1,17 @@
[
{
"_p": "bn91jc",
"chartType": "3M",
"symbol": "BBCA",
"series": {
"timeStamps": [
"2026-01-13T17:00:00Z",
"2026-01-14T17:00:00Z",
"2026-01-15T17:00:00Z"
],
"prices": [8000.0, 8075.0, 8025.0],
"startTime": "2026-01-13T17:00:00Z",
"endTime": "2026-01-15T17:00:00Z"
}
}
]