Merge pull request #3 from 0xrsydn/feat/msn-full

feat(msn): full MSN Finance endpoint coverage
This commit is contained in:
Rasyidan A F 2026-03-07 04:30:24 +07:00 committed by GitHub
commit 17e90d4601
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
21 changed files with 2492 additions and 1176 deletions

320
FEATURE_SPEC.md Normal file
View file

@ -0,0 +1,320 @@
# Feature Spec: MSN Finance Full Coverage
**Branch:** `feat/msn-full`
**Status:** Draft — pending review
**Reference:** `origin/dev/rubick` (Go implementation by rubick)
---
## Background
The Rust CLI currently supports two MSN endpoints:
- `Finance/Quotes``quote()`
- `api.msn.com/keyratios``fundamentals()`
The rubick Go project (friend's scraper) demonstrates a much wider set of MSN Finance endpoints covering equities, financials, earnings, charts, sentiment, insights, and news — all using the same public API key. This spec defines the full porting roadmap from Go → Rust.
MSN API key (public, embedded in MSN Money website):
```
0QfOX3Vn51YCzitbLaRkTTBadtWpgTN8NZLW0C1SEM
```
Base URLs:
- `https://assets.msn.com/service/` — core market data (Quotes, Charts, Equities, Earnings, Sentiment, Screener)
- `https://api.msn.com/msn/v0/pages/finance/` — extended data (keyratios, insights, newsfeed)
- `https://services.bingapis.com/contentservices-finance.hedgefunddataprovider/api/v1/` — Bing ownership data
---
## Endpoints to Implement
### P0 — Core Completeness
#### 1. `Finance/Equities` — Company Profile
- **Method:** GET
- **URL:** `{MSN_ASSETS_BASE_URL}Finance/Equities?apikey={key}&ids={id}&wrapodata=false`
- **Returns:** `EquityData` — company name, description, sector, industry, website, employees, address, officers/executives
- **CLI use:** `idx stock profile BBCA` or folded into `info` subcommand
- **Rust struct:**
```rust
pub struct EquityData {
pub id: String,
pub symbol: String,
pub short_name: String,
pub long_name: String,
pub description: String,
pub sector: String,
pub industry: String,
pub website: String,
pub employees: i64,
pub address: String,
pub city: String,
pub country: String,
pub phone: String,
pub officers: Vec<Officer>,
}
pub struct Officer {
pub name: String,
pub title: String,
pub age: Option<i32>,
pub year_born: Option<i32>,
pub total_pay: Option<i64>,
}
```
- **Complexity:** Low
---
#### 2. `Finance/Equities/financialstatements` — Financial Statements
- **Method:** GET
- **URL:** `{MSN_ASSETS_BASE_URL}Finance/Equities/financialstatements?apikey={key}&ids={id}&wrapodata=false`
- **Returns:** Balance sheet, cash flow, income statement — each as a map of `{field: value}` keyed by line item name, with period metadata (reportDate, endDate, currency, source)
- **CLI use:** `idx stock financials BBCA [--statement income|balance|cashflow]`
- **Note:** Fields are dynamic (map-based), not fixed columns — render as table with row=line item, col=period if multiple periods returned
- **Rust struct:**
```rust
pub struct FinancialStatements {
pub instrument: InstrumentInfo,
pub balance_sheet: Option<BalanceSheet>,
pub cash_flow: Option<CashFlow>,
pub income_statement: Option<IncomeStatement>,
}
pub struct BalanceSheet {
pub current_assets: HashMap<String, f64>,
pub long_term_assets: HashMap<String, f64>,
pub current_liabilities: HashMap<String, f64>,
pub equity: HashMap<String, f64>,
pub currency: String,
pub report_date: String,
pub end_date: String,
}
// Similar pattern for CashFlow (financing/investing/operating) and IncomeStatement
```
- **Complexity:** Medium (dynamic maps → table rendering)
---
### P1 — High Analyst Value
#### 3. `Finance/Events/Earnings` — Earnings History & Forecast
- **Method:** GET
- **URL:** `{MSN_ASSETS_BASE_URL}Finance/Events/Earnings?apikey={key}&ids={id}&wrapodata=false`
- **Returns:**
- `EpsLastYear`, `RevenueLastYear`
- `Forecast.annual` — 2 forward years: EpsForecast, RevenueForecast, GAAP/Normalized consensus
- `Forecast.quarterly` — next 4 quarters with same fields + EarningReleaseDate
- `History.annual` — 5 years: EpsActual, EpsSurprise, EpsSurprisePercent, RevenueActual, RevenueSurprise
- `History.quarterly` — ~12 quarters of actuals + surprises
- **CLI use:** `idx stock earnings BBCA [--forecast|--history] [--annual|--quarterly]`
- **Rust struct:**
```rust
pub struct EarningsReport {
pub eps_last_year: f64,
pub revenue_last_year: f64,
pub forecast: EarningsForecast,
pub history: EarningsHistory,
}
pub struct EarningsData {
pub eps_actual: Option<f64>,
pub eps_forecast: Option<f64>,
pub eps_surprise: Option<f64>,
pub eps_surprise_pct: Option<f64>,
pub revenue_actual: Option<f64>,
pub revenue_forecast: Option<f64>,
pub revenue_surprise: Option<f64>,
pub earning_release_date: Option<String>,
pub period_type: String, // e.g. "Q42025", "2025"
}
```
- **Complexity:** Medium (nested map keyed by period string)
---
#### 4. `Finance/Charts` — Price Chart / OHLCV History
- **Method:** GET
- **URL:** `{MSN_ASSETS_BASE_URL}Finance/Charts?apikey={key}&ids={id}&chartType={type}&wrapodata=false`
- **Chart types:** `1D`, `1W`, `1M`, `3M`, `6M`, `1Y`, `3Y`, `5Y`, `MAX`
- **Returns:** Series of `ChartPoint { time, open, high, low, close, price, volume }`
- **Note:** This unblocks the `history()` provider method — current implementation explicitly returns `Unsupported`. MSN charts don't guarantee OHLCV on all timeframes (1D is often price-only), so parse defensively.
- **CLI use:** `idx stock history BBCA --period 3M` (existing command, just needs this wired up)
- **Rust struct:**
```rust
pub struct ChartPoint {
pub time: String,
pub open: Option<f64>,
pub high: Option<f64>,
pub low: Option<f64>,
pub close: Option<f64>,
pub price: f64,
pub volume: Option<i64>,
}
```
- **Complexity:** Medium (parse series array, handle missing OHLCV gracefully)
---
### P2 — Enrichment Layer
#### 5. `Finance/SentimentBrowser` — Crowd Sentiment
- **Method:** GET
- **URL:** `{MSN_ASSETS_BASE_URL}Finance/SentimentBrowser?apikey={key}&ids={id}&wrapodata=false`
- **Returns:** Per-period sentiment stats: bullish/bearish/neutral counts, time range name (e.g., "1D", "1W", "1M")
- **CLI use:** `idx stock sentiment BBCA`
- **Rust struct:**
```rust
pub struct SentimentData {
pub symbol: String,
pub statistics: Vec<SentimentPeriod>,
}
pub struct SentimentPeriod {
pub time_range: String, // "1D", "1W", "1M"
pub bullish: i32,
pub bearish: i32,
pub neutral: i32,
}
```
- **Complexity:** Low
---
#### 6. `api.msn.com/insights` — AI-Generated Insights
- **Method:** GET
- **URL:** `{MSN_API_BASE_URL}insights?apikey={key}&ids={id}&wrapodata=false`
- **Returns:** Summary text, highlights array, risks array, last updated timestamp
- **CLI use:** `idx stock insights BBCA`
- **Rust struct:**
```rust
pub struct InsightData {
pub id: String,
pub summary: String,
pub highlights: Vec<String>,
pub risks: Vec<String>,
pub last_updated: String,
}
```
- **Complexity:** Low
---
#### 7. `MSN/Feed/me` — Stock News Feed
- **Method:** GET
- **URL:** `{MSN_API_BASE_URL}` + entity feed params with stock ID
- **Returns:** News cards: title, URL, abstract, provider name, publish time, read time
- **CLI use:** `idx stock news BBCA [--limit 10]`
- **Rust struct:**
```rust
pub struct NewsItem {
pub id: String,
pub title: String,
pub url: String,
pub description: String,
pub provider: String,
pub published_at: String,
pub read_time_min: Option<i32>,
}
```
- **Complexity:** Medium (URL construction + response parsing needs rubick reference)
---
#### 8. `Finance/Screener` — IDX Universe Screener
- **Method:** POST
- **URL:** `{MSN_ASSETS_BASE_URL}Finance/Screener?apikey={key}&wrapodata=false`
- **Body:** `{ filter: [{key, keyGroup, isRange}], order: {key, dir}, returnValueType: [...], screenerType: "...", limit: 50 }`
- **Returns:** List of stocks with quote data (price, change, market cap, volume, 52w hi/lo, YTD return)
- **CLI use:** `idx screen [--preset top-gainers|top-losers|most-active|...]`
- **Complexity:** Medium (POST body construction, preset filter definitions)
---
### P3 — Optional / Future
#### 9. Bing Ownership API — Institutional Holders
- **Base:** `https://services.bingapis.com/contentservices-finance.hedgefunddataprovider/api/v1/`
- **Endpoints:**
- `GetSecurityTopShareHolders`
- `GetSecurityTopBuyers` / `GetSecurityTopSellers`
- `GetSecurityTopNewShareHolders` / `GetSecurityTopExitedShareHolders`
- **CLI use:** `idx stock holders BBCA [--buyers|--sellers|--new|--exited]`
- **Note:** Separate base URL, may need different auth/headers than MSN. Validate working before implementing.
- **Complexity:** Low-Medium
---
## Implementation Plan
### Phase 1 — Extend `src/api/msn/`
1. Add `fetch_equities(symbol)` to `client.rs`
2. Add `fetch_financial_statements(symbol)` to `client.rs`
3. Add `fetch_earnings(symbol)` to `client.rs`
4. Add `fetch_charts(symbol, period)` to `client.rs`
5. Add corresponding parse functions to `parse.rs`
6. Expose via new methods on `MsnProvider` in `mod.rs`
### Phase 2 — New Rust structs in `src/api/msn/types.rs` (new file)
- Extract shared types (currently inline in `parse.rs`) into dedicated `types.rs`
- Add all new structs listed above
### Phase 3 — Wire CLI commands in `src/cli/stocks.rs`
New subcommands to add:
```
idx stock profile <SYMBOL> # Company info + officers
idx stock financials <SYMBOL> # Income / balance / cashflow
idx stock earnings <SYMBOL> # EPS history + forecast
idx stock sentiment <SYMBOL> # Crowd sentiment
idx stock insights <SYMBOL> # AI highlights + risks
idx stock news <SYMBOL> # News feed
idx screen # IDX screener (separate top-level command)
```
And unblock existing:
```
idx stock history <SYMBOL> # Wire MSN charts (currently Unsupported)
```
### Phase 4 — Output formatting
- Table output for financials (line item rows, period columns)
- Compact output for earnings (actual vs forecast vs surprise %)
- JSON output flag `--json` should work for all new commands
---
## Open Questions
1. **Chart OHLCV completeness** — rubick notes that MSN charts don't always return full OHLCV on short timeframes (e.g., 1D is price-only). Do we want to keep `history()` returning `Unsupported` for MSN and add a separate `charts()` method, or silently map price → close for compatibility?
2. **Financial statements period count** — The API returns one period per call (most recent). Do we want to add a bulk-fetch loop (e.g., fetch last 4 quarters separately) or just expose single-period for now?
3. **News feed URL construction** — needs exact param structure from rubick's `GetNewsFeed()` Go implementation. Worth a closer look before implementing.
4. **Screener presets** — rubick defines filter key constants (e.g., `"st_list_topperfs"`, `"st_reg_id"`). Need to decide which presets to expose as CLI flags and what the default screener view looks like.
5. **Provider trait extension**`quote()`, `fundamentals()`, `history()` are currently defined on `Provider` trait. New methods (earnings, profile, etc.) are MSN-specific — do we extend the trait or expose them as inherent methods on `MsnProvider` only?
---
## Files to Touch
```
src/api/msn/
client.rs — add fetch_* methods
mod.rs — expose new provider methods
parse.rs — add parse_* functions
types.rs — NEW: shared type definitions
src/cli/
stocks.rs — add new subcommands + output formatting
tests/
cli.rs — integration tests for new commands
fixtures/ — add response fixtures for new endpoints
```
---
*Drafted by Ciphercat based on rubick Go implementation analysis + live MSN API verification.*

View file

@ -4,17 +4,62 @@ pub mod yahoo;
use crate::config::ProviderKind; use crate::config::ProviderKind;
use crate::error::IdxError; use crate::error::IdxError;
use types::{Fundamentals, Interval, Ohlc, Period, Quote}; use types::{
Bar, CompanyProfile, EarningsReport, FinancialStatements, Fundamentals, InsightData, Interval,
NewsItem, Period, Quote, SentimentData,
};
pub trait MarketDataProvider { pub trait QuoteProvider {
fn quote(&self, symbol: &str) -> Result<Quote, IdxError>; fn quote(&self, symbol: &str) -> Result<Quote, IdxError>;
fn fundamentals(&self, symbol: &str) -> Result<Fundamentals, IdxError>; }
pub trait HistoryProvider {
fn history( fn history(
&self, &self,
symbol: &str, symbol: &str,
period: &Period, period: &Period,
interval: &Interval, interval: &Interval,
) -> Result<Vec<Ohlc>, IdxError>; ) -> Result<Vec<Bar>, IdxError>;
}
pub trait FundamentalsProvider {
fn fundamentals(&self, symbol: &str) -> Result<Fundamentals, IdxError>;
}
/// Core provider trait — quote + fundamentals only.
/// History is a separate capability (`HistoryProvider`) not all providers support
/// (e.g. MSN Finance/Charts returns 404 for IDX/XIDX stocks).
pub trait MarketDataProvider: QuoteProvider + FundamentalsProvider {}
impl<T> MarketDataProvider for T where T: QuoteProvider + FundamentalsProvider {}
#[allow(dead_code)]
pub trait ProfileProvider {
fn profile(&self, symbol: &str) -> Result<CompanyProfile, IdxError>;
}
#[allow(dead_code)]
pub trait EarningsProvider {
fn earnings(&self, symbol: &str) -> Result<EarningsReport, IdxError>;
}
#[allow(dead_code)]
pub trait FinancialsProvider {
fn financials(&self, symbol: &str) -> Result<FinancialStatements, IdxError>;
}
#[allow(dead_code)]
pub trait SentimentProvider {
fn sentiment(&self, symbol: &str) -> Result<SentimentData, IdxError>;
}
#[allow(dead_code)]
pub trait InsightsProvider {
fn insights(&self, symbol: &str) -> Result<InsightData, IdxError>;
}
#[allow(dead_code)]
pub trait NewsProvider {
fn news(&self, symbol: &str, limit: usize) -> Result<Vec<NewsItem>, IdxError>;
} }
pub fn resolve_symbol(symbol: &str, exchange: &str) -> String { pub fn resolve_symbol(symbol: &str, exchange: &str) -> String {
@ -39,10 +84,22 @@ pub fn default_provider(provider: ProviderKind, verbose: bool) -> Box<dyn Market
} }
} }
/// Returns a history-capable provider, or `None` if the selected provider doesn't
/// support price history (e.g. MSN Finance/Charts returns 404 for IDX/XIDX stocks).
pub fn history_provider(provider: ProviderKind, verbose: bool) -> Option<Box<dyn HistoryProvider>> {
if std::env::var("IDX_USE_MOCK_PROVIDER").is_ok() {
return Some(Box::new(MockProvider::from_fixtures(provider)));
}
match provider {
ProviderKind::Yahoo => Some(Box::new(yahoo::YahooProvider::new(verbose))),
ProviderKind::Msn => None,
}
}
pub struct MockProvider { pub struct MockProvider {
quote: Result<Quote, IdxError>, quote: Result<Quote, IdxError>,
fundamentals: Result<Fundamentals, IdxError>, fundamentals: Result<Fundamentals, IdxError>,
history: Result<Vec<Ohlc>, IdxError>, history: Result<Vec<Bar>, IdxError>,
} }
impl MockProvider { impl MockProvider {
@ -69,7 +126,7 @@ impl MockProvider {
.map_err(|e| IdxError::ParseError(e.to_string())); .map_err(|e| IdxError::ParseError(e.to_string()));
let fundamentals = yahoo::parse_fundamentals_from_str("BBCA.JK", &fundamentals_raw) let fundamentals = yahoo::parse_fundamentals_from_str("BBCA.JK", &fundamentals_raw)
.map_err(|e| IdxError::ParseError(e.to_string())); .map_err(|e| IdxError::ParseError(e.to_string()));
let history = yahoo::parse_history_from_str(&history_raw) let history = yahoo::parse_history_from_str("BBCA.JK", &history_raw)
.map_err(|e| IdxError::ParseError(e.to_string())); .map_err(|e| IdxError::ParseError(e.to_string()));
Self { Self {
@ -82,8 +139,6 @@ impl MockProvider {
fn from_msn_fixtures() -> Self { fn from_msn_fixtures() -> Self {
let quote_raw = std::fs::read_to_string("tests/fixtures/msn_quote_bbca.json") let quote_raw = std::fs::read_to_string("tests/fixtures/msn_quote_bbca.json")
.unwrap_or_else(|_| "[]".to_string()); .unwrap_or_else(|_| "[]".to_string());
let history_raw = std::fs::read_to_string("tests/fixtures/msn_chart_bbca_3mo.json")
.unwrap_or_else(|_| "[]".to_string());
let fundamentals_raw = std::fs::read_to_string("tests/fixtures/msn_keyratios_bbca.json") let fundamentals_raw = std::fs::read_to_string("tests/fixtures/msn_keyratios_bbca.json")
.unwrap_or_else(|_| "[]".to_string()); .unwrap_or_else(|_| "[]".to_string());
@ -91,9 +146,10 @@ impl MockProvider {
.map_err(|e| IdxError::ParseError(e.to_string())); .map_err(|e| IdxError::ParseError(e.to_string()));
let fundamentals = msn::parse_fundamentals_from_str(&fundamentals_raw, Some(&quote_raw)) let fundamentals = msn::parse_fundamentals_from_str(&fundamentals_raw, Some(&quote_raw))
.map_err(|e| IdxError::ParseError(e.to_string())); .map_err(|e| IdxError::ParseError(e.to_string()));
let history = // MSN Finance/Charts returns 404 for IDX (XIDX) — history not supported
msn::parse_history_from_str(&crate::api::types::Period::ThreeMonths, &history_raw) let history = Err(IdxError::Unsupported(
.map_err(|e| IdxError::ParseError(e.to_string())); "MSN does not provide price history for IDX stocks. Use --provider yahoo.".into(),
));
Self { Self {
quote, quote,
@ -111,23 +167,27 @@ impl MockProvider {
} }
} }
impl MarketDataProvider for MockProvider { impl QuoteProvider for MockProvider {
fn quote(&self, symbol: &str) -> Result<Quote, IdxError> { fn quote(&self, symbol: &str) -> Result<Quote, IdxError> {
let mut q = self.quote.clone()?; let mut q = self.quote.clone()?;
q.symbol = symbol.to_string(); q.symbol = symbol.to_string();
Ok(q) Ok(q)
} }
}
impl FundamentalsProvider for MockProvider {
fn fundamentals(&self, _symbol: &str) -> Result<Fundamentals, IdxError> { fn fundamentals(&self, _symbol: &str) -> Result<Fundamentals, IdxError> {
self.fundamentals.clone() self.fundamentals.clone()
} }
}
impl HistoryProvider for MockProvider {
fn history( fn history(
&self, &self,
_symbol: &str, _symbol: &str,
_period: &Period, _period: &Period,
_interval: &Interval, _interval: &Interval,
) -> Result<Vec<Ohlc>, IdxError> { ) -> Result<Vec<Bar>, IdxError> {
self.history.clone() self.history.clone()
} }
} }

View file

@ -1,10 +1,14 @@
use std::time::Duration; use std::time::Duration;
use serde::Serialize;
use serde::de::DeserializeOwned; use serde::de::DeserializeOwned;
use crate::error::IdxError; use crate::error::IdxError;
use super::parse::{KeyRatios, MsnQuote}; use super::raw_types::{
KeyRatios, MsnQuote, RawEarningsResponse, RawEquity, RawFinancialStatement, RawInsight,
RawNewsFeed, RawScreenerResponse, RawSentiment, ScreenerFilter, ScreenerOrder, ScreenerRequest,
};
use super::symbols::resolve_msn_id; use super::symbols::resolve_msn_id;
const USER_AGENT: &str = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36"; const USER_AGENT: &str = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36";
@ -34,6 +38,8 @@ impl MsnClient {
symbol: &str, symbol: &str,
endpoint: &str, endpoint: &str,
) -> Result<T, IdxError> { ) -> Result<T, IdxError> {
let mut wait = Duration::from_millis(500);
for attempt in 0..3 {
let response = self let response = self
.agent .agent
.get(url) .get(url)
@ -45,14 +51,87 @@ impl MsnClient {
.call(); .call();
match response { match response {
Ok(ok) => ok Ok(ok) => {
return ok
.into_body() .into_body()
.read_json::<T>() .read_json::<T>()
.map_err(|e| IdxError::ParseError(format!("msn {endpoint}: {e}"))), .map_err(|e| IdxError::ParseError(format!("msn {endpoint}: {e}")));
Err(ureq::Error::StatusCode(404)) => Err(IdxError::SymbolNotFound(symbol.to_string())),
Err(ureq::Error::StatusCode(429)) => Err(IdxError::RateLimited),
Err(err) => Err(IdxError::Http(format!("msn {endpoint}: {err}"))),
} }
Err(ureq::Error::StatusCode(404)) => {
return Err(IdxError::SymbolNotFound(symbol.to_string()));
}
Err(ureq::Error::StatusCode(429)) => {
if attempt < 2 {
std::thread::sleep(wait);
wait *= 2;
continue;
}
return Err(IdxError::RateLimited);
}
Err(ureq::Error::StatusCode(code)) if code >= 500 => {
if attempt < 2 {
std::thread::sleep(wait);
wait *= 2;
continue;
}
return Err(IdxError::Http(format!("msn {endpoint}: status {code}")));
}
Err(err) => return Err(IdxError::Http(format!("msn {endpoint}: {err}"))),
}
}
Err(IdxError::RateLimited)
}
fn post_json<B: Serialize, T: DeserializeOwned>(
&self,
url: &str,
body: &B,
symbol: &str,
endpoint: &str,
) -> Result<T, IdxError> {
let mut wait = Duration::from_millis(500);
for attempt in 0..3 {
let response = self
.agent
.post(url)
.header("User-Agent", USER_AGENT)
.header("Accept", "application/json")
.header("Accept-Language", "en-US,en;q=0.9,id;q=0.8")
.header("Origin", "https://www.msn.com")
.header("Referer", "https://www.msn.com/")
.header("Content-Type", "text/plain;charset=UTF-8")
.send_json(body);
match response {
Ok(ok) => {
return ok
.into_body()
.read_json::<T>()
.map_err(|e| IdxError::ParseError(format!("msn {endpoint}: {e}")));
}
Err(ureq::Error::StatusCode(404)) => {
return Err(IdxError::SymbolNotFound(symbol.to_string()));
}
Err(ureq::Error::StatusCode(429)) => {
if attempt < 2 {
std::thread::sleep(wait);
wait *= 2;
continue;
}
return Err(IdxError::RateLimited);
}
Err(ureq::Error::StatusCode(code)) if code >= 500 => {
if attempt < 2 {
std::thread::sleep(wait);
wait *= 2;
continue;
}
return Err(IdxError::Http(format!("msn {endpoint}: status {code}")));
}
Err(err) => return Err(IdxError::Http(format!("msn {endpoint}: {err}"))),
}
}
Err(IdxError::RateLimited)
} }
pub(super) fn fetch_quotes(&self, symbol: &str) -> Result<Vec<MsnQuote>, IdxError> { pub(super) fn fetch_quotes(&self, symbol: &str) -> Result<Vec<MsnQuote>, IdxError> {
@ -71,4 +150,94 @@ impl MsnClient {
format!("{MSN_API_BASE_URL}keyratios?apikey={MSN_API_KEY}&ids={id}&wrapodata=false"); format!("{MSN_API_BASE_URL}keyratios?apikey={MSN_API_KEY}&ids={id}&wrapodata=false");
self.get_json(&url, symbol, "keyratios") self.get_json(&url, symbol, "keyratios")
} }
pub(super) fn fetch_equities(&self, symbol: &str) -> Result<Vec<RawEquity>, IdxError> {
let id =
resolve_msn_id(symbol).ok_or_else(|| IdxError::SymbolNotFound(symbol.to_string()))?;
let url = format!(
"{MSN_ASSETS_BASE_URL}Finance/Equities?apikey={MSN_API_KEY}&ids={id}&wrapodata=false"
);
self.get_json(&url, symbol, "equities")
}
pub(super) fn fetch_financial_statements(
&self,
symbol: &str,
) -> Result<Vec<RawFinancialStatement>, IdxError> {
let id =
resolve_msn_id(symbol).ok_or_else(|| IdxError::SymbolNotFound(symbol.to_string()))?;
let url = format!(
"{MSN_ASSETS_BASE_URL}Finance/Equities/financialstatements?apikey={MSN_API_KEY}&ids={id}&wrapodata=false"
);
self.get_json(&url, symbol, "financialstatements")
}
pub(super) fn fetch_earnings(&self, symbol: &str) -> Result<RawEarningsResponse, IdxError> {
let id =
resolve_msn_id(symbol).ok_or_else(|| IdxError::SymbolNotFound(symbol.to_string()))?;
let url = format!(
"{MSN_ASSETS_BASE_URL}Finance/Events/Earnings?apikey={MSN_API_KEY}&ids={id}&wrapodata=false"
);
self.get_json(&url, symbol, "earnings")
}
pub(super) fn fetch_sentiment(&self, symbol: &str) -> Result<Vec<RawSentiment>, IdxError> {
let id =
resolve_msn_id(symbol).ok_or_else(|| IdxError::SymbolNotFound(symbol.to_string()))?;
let url = format!(
"{MSN_ASSETS_BASE_URL}Finance/SentimentBrowser?apikey={MSN_API_KEY}&cm=id-id&it=web&scn=ANON&ids={id}&wrapodata=false&flightId=INeedDau"
);
self.get_json(&url, symbol, "sentiment")
}
pub(super) fn fetch_insights(&self, symbol: &str) -> Result<Vec<RawInsight>, IdxError> {
let id =
resolve_msn_id(symbol).ok_or_else(|| IdxError::SymbolNotFound(symbol.to_string()))?;
let url =
format!("{MSN_API_BASE_URL}insights?apikey={MSN_API_KEY}&ids={id}&wrapodata=false");
self.get_json(&url, symbol, "insights")
}
pub(super) fn fetch_news(&self, symbol: &str, limit: usize) -> Result<RawNewsFeed, IdxError> {
let id =
resolve_msn_id(symbol).ok_or_else(|| IdxError::SymbolNotFound(symbol.to_string()))?;
let url = format!(
"{MSN_ASSETS_BASE_URL}MSN/Feed/me?$top={limit}&apikey={MSN_API_KEY}&cm=id-id&contentType=article,video,slideshow&it=web&query=ef_stock_{id}&queryType=entityfeed&responseSchema=cardview&scn=ANON&wrapodata=false"
);
self.get_json(&url, symbol, "news")
}
pub(super) fn fetch_screener(
&self,
filter: &str,
region: &str,
limit: usize,
) -> Result<RawScreenerResponse, IdxError> {
let url =
format!("{MSN_ASSETS_BASE_URL}Finance/Screener?apikey={MSN_API_KEY}&wrapodata=false");
let req = ScreenerRequest {
filter: vec![
ScreenerFilter {
key: filter.to_string(),
key_group: "st_list_".to_string(),
is_range: false,
},
ScreenerFilter {
key: region.to_string(),
key_group: "st_reg_".to_string(),
is_range: false,
},
],
order: ScreenerOrder {
key: "st_1yr_asc_order".to_string(),
dir: "desc".to_string(),
},
return_value_type: vec!["quote".to_string(), "equity".to_string()],
screener_type: "stock".to_string(),
limit,
page_index: 0,
};
self.post_json(&url, &req, "SCREENER", "screener")
}
} }

521
src/api/msn/map.rs Normal file
View file

@ -0,0 +1,521 @@
use super::raw_types::{
IndustryMetric, KeyRatios, MsnQuote, RawEarningsData, RawEarningsResponse, RawEquity,
RawFinancialStatement, RawInsight, RawNewsFeed, RawScreenerResponse, RawSentiment,
RawStatementSection,
};
use super::symbols::{normalized_symbol, ticker_from_symbol};
use crate::api::types::{
CompanyProfile, EarningsData, EarningsReport, FinancialStatements, Fundamentals, InsightData,
InstrumentInfo, NewsItem, Officer, Quote, SentimentData, SentimentPeriod, StatementSection,
};
use crate::error::IdxError;
pub(super) fn parse_quote(symbol: &str, quotes: &[MsnQuote]) -> Result<Quote, IdxError> {
let quote = quotes.first().ok_or(IdxError::ProviderUnavailable)?;
let raw_price = quote
.price
.ok_or_else(|| IdxError::SymbolNotFound(symbol.to_string()))?;
let prev_close = quote.price_previous_close.map(round_price);
let price = round_price(raw_price);
let change = prev_close
.map(|previous| price - previous)
.or_else(|| quote.price_change.map(round_price))
.unwrap_or(0);
let ticker = quote
.symbol
.as_deref()
.and_then(ticker_from_symbol)
.unwrap_or_else(|| ticker_from_symbol(symbol).unwrap_or_default());
let (week52_position, range_signal) = match (quote.price_52w_low, quote.price_52w_high) {
(Some(low), Some(high)) if high > low => {
let position = (raw_price - low) / (high - low);
let signal = if position > 0.66 {
Some("upper".to_string())
} else if position < 0.33 {
Some("lower".to_string())
} else {
Some("middle".to_string())
};
(Some(position), signal)
}
_ => (None, None),
};
Ok(Quote {
symbol: normalized_symbol(symbol, &ticker),
price,
change,
change_pct: quote.price_change_percent.unwrap_or(0.0),
volume: round_u64(quote.accumulated_volume).unwrap_or(0),
market_cap: round_u64(quote.market_cap),
week52_high: quote.price_52w_high.map(round_price),
week52_low: quote.price_52w_low.map(round_price),
week52_position,
range_signal,
prev_close,
avg_volume: round_u64(quote.average_volume),
})
}
pub(super) fn parse_fundamentals(
ratios: &[KeyRatios],
quote: Option<&MsnQuote>,
) -> Result<Fundamentals, IdxError> {
let ratios = ratios.first().ok_or(IdxError::ProviderUnavailable)?;
let metrics = if ratios.company_metrics.is_empty() {
&ratios.industry_metrics
} else {
&ratios.company_metrics
};
if preferred_metric(metrics).is_none() {
return Err(IdxError::ProviderUnavailable);
}
Ok(Fundamentals {
trailing_pe: best_metric_value(metrics, |metric| metric.price_to_earnings_ratio),
forward_pe: best_metric_value(metrics, |metric| metric.forward_price_to_eps),
price_to_book: best_metric_value(metrics, |metric| metric.price_to_book_ratio),
return_on_equity: best_metric_value(metrics, |metric| normalize_percentish(metric.roe)),
profit_margins: best_metric_value(metrics, |metric| {
normalize_percentish(metric.profit_margin.or(metric.net_margin))
}),
return_on_assets: best_metric_value(metrics, |metric| {
normalize_percentish(metric.roa_ttm.or(metric.return_on_asset_current))
}),
revenue_growth: best_metric_value(metrics, |metric| {
normalize_percentish(metric.revenue_ytd_ytd.or(metric.revenue_growth_rate))
}),
earnings_growth: best_metric_value(metrics, |metric| {
normalize_percentish(
metric
.net_income_ytd_ytd_growth_rate
.or(metric.earnings_growth_rate),
)
}),
debt_to_equity: best_metric_value(metrics, |metric| metric.debt_to_equity_ratio),
current_ratio: best_metric_value(metrics, |metric| {
sanitize_current_ratio(metric.current_ratio)
}),
enterprise_value: None,
ebitda: None,
market_cap: quote.and_then(|item| round_u64(item.market_cap)),
})
}
fn preferred_metric(metrics: &[IndustryMetric]) -> Option<&IndustryMetric> {
metrics.iter().max_by_key(|metric| metric_rank(metric))
}
fn best_metric_value<T: Copy>(
metrics: &[IndustryMetric],
extractor: impl Fn(&IndustryMetric) -> Option<T>,
) -> Option<T> {
metrics
.iter()
.filter_map(|metric| extractor(metric).map(|value| (metric_rank(metric), value)))
.max_by_key(|(rank, _)| *rank)
.map(|(_, value)| value)
}
fn metric_rank(metric: &IndustryMetric) -> (i32, i32) {
(
metric
.year
.as_deref()
.and_then(|year| year.parse::<i32>().ok())
.unwrap_or(i32::MIN),
metric_period_priority(metric.fiscal_period_type.as_deref()),
)
}
fn metric_period_priority(period: Option<&str>) -> i32 {
match period.map(|value| value.trim()) {
Some(value) if value.eq_ignore_ascii_case("TTM") => 7,
Some(value)
if value.eq_ignore_ascii_case("ANNUAL")
|| value.eq_ignore_ascii_case("FY")
|| value.eq_ignore_ascii_case("YEAR") =>
{
6
}
Some(value) if value.eq_ignore_ascii_case("Q4") => 5,
Some(value) if value.eq_ignore_ascii_case("Q3") => 4,
Some(value) if value.eq_ignore_ascii_case("Q2") => 3,
Some(value) if value.eq_ignore_ascii_case("Q1") => 2,
Some(value) if value.eq_ignore_ascii_case("NTM") => 1,
_ => 0,
}
}
fn normalize_percentish(value: Option<f64>) -> Option<f64> {
value.and_then(|number| {
if !number.is_finite() {
None
} else if number.abs() > 1.0 {
Some(number / 100.0)
} else {
Some(number)
}
})
}
fn sanitize_current_ratio(value: Option<f64>) -> Option<f64> {
value.and_then(|number| {
if !number.is_finite() || number < 0.01 {
None
} else {
Some(number)
}
})
}
fn round_price(value: f64) -> i64 {
value.round() as i64
}
fn round_u64(value: Option<f64>) -> Option<u64> {
value.and_then(|number| {
if !number.is_finite() || number.is_sign_negative() {
None
} else {
Some(number.round() as u64)
}
})
}
pub(super) fn parse_profile(symbol: &str, raw: &[RawEquity]) -> Result<CompanyProfile, IdxError> {
let equity = raw
.first()
.ok_or_else(|| IdxError::ParseError("no profile data".into()))?;
Ok(CompanyProfile {
id: equity.id.clone().unwrap_or_default(),
symbol: equity.symbol.clone().unwrap_or_else(|| symbol.to_string()),
short_name: equity.short_name.clone().unwrap_or_default(),
long_name: equity.long_name.clone().unwrap_or_default(),
description: equity.description.clone().unwrap_or_default(),
sector: equity.sector.clone().unwrap_or_default(),
industry: equity.industry.clone().unwrap_or_default(),
website: equity.website.clone().unwrap_or_default(),
employees: equity.full_time_employees.unwrap_or_default(),
address: equity.address.clone().unwrap_or_default(),
city: equity.city.clone().unwrap_or_default(),
country: equity.country.clone().unwrap_or_default(),
phone: equity.phone.clone().unwrap_or_default(),
officers: equity
.officers
.as_ref()
.map(|items| {
items
.iter()
.map(|officer| Officer {
name: officer.name.clone().unwrap_or_default(),
title: officer.title.clone().unwrap_or_default(),
age: officer.age,
year_born: officer.year_born,
total_pay: officer.total_pay,
})
.collect()
})
.unwrap_or_default(),
})
}
pub(super) fn parse_financial_statements(
symbol: &str,
raw: &[RawFinancialStatement],
) -> Result<FinancialStatements, IdxError> {
let item = raw
.first()
.ok_or_else(|| IdxError::ParseError("no financial statements".into()))?;
let instrument = item.underlying_instrument.as_ref();
Ok(FinancialStatements {
instrument: InstrumentInfo {
id: instrument
.and_then(|v| v.instrument_id.clone())
.unwrap_or_default(),
symbol: instrument
.and_then(|v| v.symbol.clone())
.unwrap_or_else(|| symbol.to_string()),
name: instrument
.and_then(|v| v.display_name.clone().or_else(|| v.short_name.clone()))
.unwrap_or_default(),
},
balance_sheet: item.balance_sheets.as_ref().map(parse_statement_section),
cash_flow: item.cash_flow.as_ref().map(parse_statement_section),
income_statement: item.income_statements.as_ref().map(parse_statement_section),
})
}
pub(super) fn parse_earnings(
_symbol: &str,
raw: &RawEarningsResponse,
) -> Result<EarningsReport, IdxError> {
let mut forecast = Vec::new();
let mut history = Vec::new();
if let Some(bucket) = &raw.forecast {
collect_earnings(bucket.annual.as_ref(), &mut forecast);
collect_earnings(bucket.quarterly.as_ref(), &mut forecast);
}
if let Some(bucket) = &raw.history {
collect_earnings(bucket.annual.as_ref(), &mut history);
collect_earnings(bucket.quarterly.as_ref(), &mut history);
}
forecast.sort_by_key(|row| row.earning_release_date.clone().unwrap_or_default());
history.sort_by_key(|row| row.earning_release_date.clone().unwrap_or_default());
Ok(EarningsReport {
eps_last_year: raw.eps_last_year.unwrap_or_default(),
revenue_last_year: raw.revenue_last_year.unwrap_or_default(),
forecast,
history,
})
}
pub(super) fn parse_sentiment(
symbol: &str,
raw: &[RawSentiment],
) -> Result<SentimentData, IdxError> {
let item = raw
.first()
.ok_or_else(|| IdxError::ParseError("no sentiment data".into()))?;
let stats = item
.sentiment_statistics
.as_ref()
.map(|items| {
items
.iter()
.map(|it| SentimentPeriod {
time_range: it.time_range_name.clone().unwrap_or_default(),
bullish: it.bullish.unwrap_or_default(),
bearish: it.bearish.unwrap_or_default(),
neutral: it.neutral.unwrap_or_default(),
})
.collect()
})
.unwrap_or_default();
Ok(SentimentData {
symbol: item.symbol.clone().unwrap_or_else(|| symbol.to_string()),
statistics: stats,
})
}
pub(super) fn parse_insights(symbol: &str, raw: &[RawInsight]) -> Result<InsightData, IdxError> {
let item = raw
.first()
.ok_or_else(|| IdxError::ParseError("no insights data".into()))?;
let insights = item.insights.as_deref().unwrap_or(&[]);
// Group insight statements into highlights (non-risk) and risks by category
let highlights: Vec<String> = insights
.iter()
.filter(|i| {
i.category
.as_deref()
.map(|c| !c.eq_ignore_ascii_case("risk"))
.unwrap_or(true)
})
.filter_map(|i| {
let name = i.insight_name.as_deref().unwrap_or("");
let stmt = i.insight_statement.as_deref().unwrap_or("");
if stmt.is_empty() {
None
} else if name.is_empty() {
Some(stmt.to_string())
} else {
Some(format!("{name}: {stmt}"))
}
})
.collect();
let risks: Vec<String> = insights
.iter()
.filter(|i| {
i.category
.as_deref()
.map(|c| c.eq_ignore_ascii_case("risk"))
.unwrap_or(false)
})
.filter_map(|i| {
let stmt = i.insight_statement.as_deref().unwrap_or("");
if stmt.is_empty() {
None
} else {
Some(stmt.to_string())
}
})
.collect();
Ok(InsightData {
id: item
.instrument_id
.clone()
.unwrap_or_else(|| symbol.to_string()),
summary: item.display_name.clone().unwrap_or_default(),
highlights,
risks,
last_updated: String::new(),
})
}
pub(super) fn parse_news(raw: &RawNewsFeed) -> Result<Vec<NewsItem>, IdxError> {
let source = raw
.sub_cards
.as_ref()
.or(raw.value.as_ref())
.ok_or_else(|| IdxError::ParseError("no news data".into()))?;
Ok(source
.iter()
.map(|item| NewsItem {
id: item.id.clone().unwrap_or_default(),
title: item.title.clone().unwrap_or_default(),
url: item.url.clone().unwrap_or_default(),
description: item.description.clone().unwrap_or_default(),
provider: item
.provider
.as_ref()
.and_then(|p| p.name.clone())
.unwrap_or_default(),
published_at: item.published_date_time.clone().unwrap_or_default(),
read_time_min: item.read_time_min,
})
.collect())
}
pub(super) fn parse_screener_results(raw: &RawScreenerResponse) -> Result<Vec<Quote>, IdxError> {
let quotes = raw
.quote
.as_ref()
.ok_or_else(|| IdxError::ParseError("no screener data".into()))?;
// Build Quote directly from screener MsnQuote data; skip stocks with no price
// (do not route through parse_quote which errors on missing price)
let results: Vec<Quote> = quotes
.iter()
.filter_map(|q| {
let raw_price = q.price?; // skip if no price
let price = round_price(raw_price);
let prev_close = q.price_previous_close.map(round_price);
let change = prev_close
.map(|pc| price - pc)
.or_else(|| q.price_change.map(round_price))
.unwrap_or(0);
let ticker = q
.symbol
.as_deref()
.and_then(ticker_from_symbol)
.unwrap_or_default();
let (week52_position, range_signal) = match (q.price_52w_low, q.price_52w_high) {
(Some(low), Some(high)) if high > low => {
let pos = (raw_price - low) / (high - low);
let sig = if pos > 0.66 {
"upper"
} else if pos < 0.33 {
"lower"
} else {
"middle"
};
(Some(pos), Some(sig.to_string()))
}
_ => (None, None),
};
Some(Quote {
symbol: normalized_symbol(&ticker, &ticker),
price,
change,
change_pct: q.price_change_percent.unwrap_or(0.0),
volume: round_u64(q.accumulated_volume).unwrap_or(0),
market_cap: round_u64(q.market_cap),
week52_high: q.price_52w_high.map(round_price),
week52_low: q.price_52w_low.map(round_price),
week52_position,
range_signal,
prev_close,
avg_volume: round_u64(q.average_volume),
})
})
.collect();
if results.is_empty() {
return Err(IdxError::ParseError(
"screener returned no priced stocks".into(),
));
}
Ok(results)
}
fn parse_statement_section(section: &RawStatementSection) -> StatementSection {
// MSN financial statement values are nested one level deep inside sub-objects
// (e.g., incomeStatement.income.{lineItems}, incomeStatement.revenue.{lineItems})
// Flatten all numeric values from any depth-1 sub-object into a single map.
let skip_keys = [
"currency",
"source",
"sourceDate",
"reportDate",
"endDate",
"fiscalYearEndMonth",
"statementType",
"type",
"_p",
"_t",
"year",
"underlyingInstrument",
"id",
];
let mut values = std::collections::HashMap::new();
for (k, v) in &section.data {
if skip_keys.contains(&k.as_str()) {
continue;
}
if let Some(num) = v.as_f64() {
// Direct numeric value at top level
values.insert(k.to_string(), num);
} else if let Some(obj) = v.as_object() {
// Nested sub-object — flatten one level (e.g., income.{lineItem: value})
for (sub_k, sub_v) in obj {
if let Some(num) = sub_v.as_f64() {
values.insert(sub_k.to_string(), num);
}
}
}
}
StatementSection {
values,
currency: section.currency.clone().unwrap_or_default(),
report_date: section.report_date.clone().unwrap_or_default(),
end_date: section.end_date.clone().unwrap_or_default(),
}
}
fn collect_earnings(
values: Option<&std::collections::HashMap<String, RawEarningsData>>,
out: &mut Vec<EarningsData>,
) {
let Some(values) = values else {
return;
};
let mut rows: Vec<(&String, &RawEarningsData)> = values.iter().collect();
rows.sort_by_key(|(k, _)| (*k).clone());
for (_, v) in rows {
out.push(EarningsData {
eps_actual: v.eps_actual,
eps_forecast: v.eps_forecast,
eps_surprise: v.eps_surprise,
eps_surprise_pct: v.eps_surprise_percent,
revenue_actual: v.revenue_actual,
revenue_forecast: v.revenue_forecast,
revenue_surprise: v.revenue_surprise,
earning_release_date: v.earning_release_date.clone(),
period_type: v.ciq_fiscal_period_type.clone().unwrap_or_default(),
});
}
}

View file

@ -1,83 +1,102 @@
mod client; mod client;
mod map;
mod parse; mod parse;
mod raw_types;
mod symbols; mod symbols;
use crate::api::MarketDataProvider; use crate::api::types::{
use crate::api::types::{Fundamentals, Interval, Ohlc, Period, Quote}; CompanyProfile, EarningsReport, FinancialStatements, Fundamentals, InsightData, NewsItem,
Quote, SentimentData,
};
use crate::api::{
EarningsProvider, FinancialsProvider, FundamentalsProvider, InsightsProvider, NewsProvider,
ProfileProvider, QuoteProvider, SentimentProvider,
};
use crate::error::IdxError; use crate::error::IdxError;
use client::MsnClient; use client::MsnClient;
use parse::{parse_fundamentals, parse_quote}; use map::{
parse_earnings, parse_financial_statements, parse_fundamentals, parse_insights, parse_news,
parse_profile, parse_quote, parse_screener_results, parse_sentiment,
};
pub(crate) use parse::{parse_fundamentals_from_str, parse_history_from_str, parse_quote_from_str}; pub(crate) use parse::{parse_fundamentals_from_str, parse_quote_from_str};
const HISTORY_UNSUPPORTED_REASON: &str = "MSN provider does not currently support history or technical analysis because MSN charts do not consistently expose real OHLCV data";
pub struct MsnProvider { pub struct MsnProvider {
client: MsnClient, client: MsnClient,
verbose: bool,
} }
impl MsnProvider { impl MsnProvider {
pub fn new(verbose: bool) -> Self { pub fn new(_verbose: bool) -> Self {
Self { Self {
client: MsnClient::new(), client: MsnClient::new(),
verbose,
} }
} }
pub fn screener(
&self,
filter: &str,
region: &str,
limit: usize,
) -> Result<Vec<Quote>, IdxError> {
let raw = self.client.fetch_screener(filter, region, limit)?;
parse_screener_results(&raw)
}
} }
impl MarketDataProvider for MsnProvider { impl QuoteProvider for MsnProvider {
fn quote(&self, symbol: &str) -> Result<Quote, IdxError> { fn quote(&self, symbol: &str) -> Result<Quote, IdxError> {
let quotes = self.client.fetch_quotes(symbol)?; let quotes = self.client.fetch_quotes(symbol)?;
parse_quote(symbol, &quotes) parse_quote(symbol, &quotes)
} }
}
impl FundamentalsProvider for MsnProvider {
fn fundamentals(&self, symbol: &str) -> Result<Fundamentals, IdxError> { fn fundamentals(&self, symbol: &str) -> Result<Fundamentals, IdxError> {
let ratios = self.client.fetch_key_ratios(symbol)?; let ratios = self.client.fetch_key_ratios(symbol)?;
let quote = self let quote = self.client.fetch_quotes(symbol)?;
.client parse_fundamentals(&ratios, quote.first())
.fetch_quotes(symbol)
.map_err(|e| {
if self.verbose {
eprintln!("warning: quote fetch for fundamentals failed: {e}");
}
e
})
.ok()
.and_then(|quotes| quotes.into_iter().next());
parse_fundamentals(&ratios, quote.as_ref())
}
fn history(
&self,
_symbol: &str,
_period: &Period,
_interval: &Interval,
) -> Result<Vec<Ohlc>, IdxError> {
Err(IdxError::Unsupported(
HISTORY_UNSUPPORTED_REASON.to_string(),
))
} }
} }
#[cfg(test)] impl ProfileProvider for MsnProvider {
mod tests { fn profile(&self, symbol: &str) -> Result<CompanyProfile, IdxError> {
use super::MsnProvider; let raw = self.client.fetch_equities(symbol)?;
use crate::api::MarketDataProvider; parse_profile(symbol, &raw)
use crate::api::types::{Interval, Period}; }
use crate::error::IdxError; }
#[test] impl EarningsProvider for MsnProvider {
fn history_is_explicitly_unsupported() { fn earnings(&self, symbol: &str) -> Result<EarningsReport, IdxError> {
let provider = MsnProvider::new(false); let raw = self.client.fetch_earnings(symbol)?;
let err = provider parse_earnings(symbol, &raw)
.history("BBCA.JK", &Period::OneMonth, &Interval::Day) }
.expect_err("history should be unsupported"); }
assert!(matches!(err, IdxError::Unsupported(_)));
assert!( impl FinancialsProvider for MsnProvider {
err.to_string() fn financials(&self, symbol: &str) -> Result<FinancialStatements, IdxError> {
.contains("MSN provider does not currently support history or technical analysis") let raw = self.client.fetch_financial_statements(symbol)?;
); parse_financial_statements(symbol, &raw)
}
}
impl SentimentProvider for MsnProvider {
fn sentiment(&self, symbol: &str) -> Result<SentimentData, IdxError> {
let raw = self.client.fetch_sentiment(symbol)?;
parse_sentiment(symbol, &raw)
}
}
impl InsightsProvider for MsnProvider {
fn insights(&self, symbol: &str) -> Result<InsightData, IdxError> {
let raw = self.client.fetch_insights(symbol)?;
parse_insights(symbol, &raw)
}
}
impl NewsProvider for MsnProvider {
fn news(&self, symbol: &str, limit: usize) -> Result<Vec<NewsItem>, IdxError> {
let raw = self.client.fetch_news(symbol, limit)?;
parse_news(&raw)
} }
} }

View file

@ -1,11 +1,6 @@
use std::collections::BTreeMap; use super::map::{parse_fundamentals, parse_quote};
use super::raw_types::{KeyRatios, MsnQuote};
use chrono::{Datelike, NaiveDate}; use crate::api::types::{Fundamentals, Quote};
use serde::de::Error as _;
use serde::{Deserialize, Deserializer};
use super::symbols::{normalized_symbol, ticker_from_symbol};
use crate::api::types::{Fundamentals, Ohlc, Period, Quote};
use crate::error::IdxError; use crate::error::IdxError;
#[cfg_attr(not(test), allow(dead_code))] #[cfg_attr(not(test), allow(dead_code))]
@ -15,55 +10,6 @@ pub(crate) fn parse_quote_from_str(symbol: &str, raw: &str) -> Result<Quote, Idx
parse_quote(symbol, &quotes) parse_quote(symbol, &quotes)
} }
pub(super) fn parse_quote(symbol: &str, quotes: &[MsnQuote]) -> Result<Quote, IdxError> {
let quote = quotes.first().ok_or(IdxError::ProviderUnavailable)?;
let raw_price = quote
.price
.ok_or_else(|| IdxError::SymbolNotFound(symbol.to_string()))?;
let prev_close = quote.price_previous_close.map(round_price);
let price = round_price(raw_price);
let change = prev_close
.map(|previous| price - previous)
.or_else(|| quote.price_change.map(round_price))
.unwrap_or(0);
let ticker = quote
.symbol
.as_deref()
.and_then(ticker_from_symbol)
.unwrap_or_else(|| ticker_from_symbol(symbol).unwrap_or_default());
let (week52_position, range_signal) = match (quote.price_52w_low, quote.price_52w_high) {
(Some(low), Some(high)) if high > low => {
let position = (raw_price - low) / (high - low);
let signal = if position > 0.66 {
Some("upper".to_string())
} else if position < 0.33 {
Some("lower".to_string())
} else {
Some("middle".to_string())
};
(Some(position), signal)
}
_ => (None, None),
};
Ok(Quote {
symbol: normalized_symbol(symbol, &ticker),
price,
change,
change_pct: quote.price_change_percent.unwrap_or(0.0),
volume: round_u64(quote.accumulated_volume).unwrap_or(0),
market_cap: round_u64(quote.market_cap),
week52_high: quote.price_52w_high.map(round_price),
week52_low: quote.price_52w_low.map(round_price),
week52_position,
range_signal,
prev_close,
avg_volume: round_u64(quote.average_volume),
})
}
#[cfg_attr(not(test), allow(dead_code))] #[cfg_attr(not(test), allow(dead_code))]
pub(crate) fn parse_fundamentals_from_str( pub(crate) fn parse_fundamentals_from_str(
raw: &str, raw: &str,
@ -79,478 +25,12 @@ pub(crate) fn parse_fundamentals_from_str(
parse_fundamentals(&ratios, quote.as_ref()) parse_fundamentals(&ratios, quote.as_ref())
} }
pub(super) fn parse_fundamentals(
ratios: &[KeyRatios],
quote: Option<&MsnQuote>,
) -> Result<Fundamentals, IdxError> {
let ratios = ratios.first().ok_or(IdxError::ProviderUnavailable)?;
let metrics = if ratios.company_metrics.is_empty() {
&ratios.industry_metrics
} else {
&ratios.company_metrics
};
if preferred_metric(metrics).is_none() {
return Err(IdxError::ProviderUnavailable);
}
Ok(Fundamentals {
trailing_pe: best_metric_value(metrics, |metric| metric.price_to_earnings_ratio),
forward_pe: best_metric_value(metrics, |metric| metric.forward_price_to_eps),
price_to_book: best_metric_value(metrics, |metric| metric.price_to_book_ratio),
return_on_equity: best_metric_value(metrics, |metric| normalize_percentish(metric.roe)),
profit_margins: best_metric_value(metrics, |metric| {
normalize_percentish(metric.profit_margin.or(metric.net_margin))
}),
return_on_assets: best_metric_value(metrics, |metric| {
normalize_percentish(metric.roa_ttm.or(metric.return_on_asset_current))
}),
revenue_growth: best_metric_value(metrics, |metric| {
normalize_percentish(metric.revenue_ytd_ytd.or(metric.revenue_growth_rate))
}),
earnings_growth: best_metric_value(metrics, |metric| {
normalize_percentish(
metric
.net_income_ytd_ytd_growth_rate
.or(metric.earnings_growth_rate),
)
}),
debt_to_equity: best_metric_value(metrics, |metric| metric.debt_to_equity_ratio),
current_ratio: best_metric_value(metrics, |metric| {
sanitize_current_ratio(metric.current_ratio)
}),
enterprise_value: None,
ebitda: None,
market_cap: quote.and_then(|item| round_u64(item.market_cap)),
})
}
#[cfg_attr(not(test), allow(dead_code))] #[cfg_attr(not(test), allow(dead_code))]
pub(crate) fn parse_history_from_str(period: &Period, raw: &str) -> Result<Vec<Ohlc>, IdxError> {
let charts: Vec<MsnChart> =
serde_json::from_str(raw).map_err(|e| IdxError::ParseError(e.to_string()))?;
parse_history_with_verbose(period, &charts, false)
}
#[allow(dead_code)] #[allow(dead_code)]
fn parse_close_only_history_from_str(
period: &Period,
raw: &str,
) -> Result<Vec<ClosePoint>, IdxError> {
let charts: Vec<MsnChart> =
serde_json::from_str(raw).map_err(|e| IdxError::ParseError(e.to_string()))?;
parse_close_only_history(period, &charts)
}
pub(super) fn parse_history_with_verbose(
period: &Period,
charts: &[MsnChart],
verbose: bool,
) -> Result<Vec<Ohlc>, IdxError> {
let chart = charts.first().ok_or(IdxError::ProviderUnavailable)?;
if !chart.series.has_real_ohlcv() {
return Err(IdxError::ParseError(
"msn does not expose real OHLC/volume for this history range".to_string(),
));
}
let timestamps = &chart.series.time_stamps;
let mut grouped: BTreeMap<NaiveDate, Ohlc> = BTreeMap::new();
let mut dropped = 0usize;
for (idx, raw_ts) in timestamps.iter().enumerate() {
let Some(date) = parse_chart_date(raw_ts) else {
dropped += 1;
continue;
};
let point = (
chart.series.open_prices.get(idx).copied(),
chart.series.prices_high.get(idx).copied(),
chart.series.prices_low.get(idx).copied(),
chart.series.prices.get(idx).copied(),
chart.series.volumes.get(idx).copied(),
);
let (Some(open), Some(high), Some(low), Some(close), Some(volume)) = point else {
dropped += 1;
continue;
};
let candle = Ohlc {
date,
open: round_price(open),
high: round_price(high),
low: round_price(low),
close: round_price(close),
volume: round_u64(Some(volume)).unwrap_or(0),
};
grouped
.entry(date)
.and_modify(|existing| {
existing.high = existing.high.max(candle.high);
existing.low = existing.low.min(candle.low);
existing.close = candle.close;
existing.volume = existing.volume.saturating_add(candle.volume);
})
.or_insert(candle);
}
let mut out: Vec<Ohlc> = grouped.into_values().collect();
trim_history_to_period(period, &mut out);
if dropped > 0 && verbose {
eprintln!("warning: dropped {dropped} OHLC row(s) from MSN response due to missing fields");
}
if out.is_empty() {
return Err(IdxError::ProviderUnavailable);
}
Ok(out)
}
fn parse_close_only_history(
period: &Period,
charts: &[MsnChart],
) -> Result<Vec<ClosePoint>, IdxError> {
let chart = charts.first().ok_or(IdxError::ProviderUnavailable)?;
let timestamps = &chart.series.time_stamps;
let mut grouped: BTreeMap<NaiveDate, ClosePoint> = BTreeMap::new();
for (idx, raw_ts) in timestamps.iter().enumerate() {
let Some(date) = parse_chart_date(raw_ts) else {
continue;
};
let Some(close) = chart.series.prices.get(idx).copied() else {
continue;
};
grouped.insert(
date,
ClosePoint {
date,
close: round_price(close),
},
);
}
let mut out: Vec<ClosePoint> = grouped.into_values().collect();
trim_close_history_to_period(period, &mut out);
if out.is_empty() {
return Err(IdxError::ProviderUnavailable);
}
Ok(out)
}
fn trim_history_to_period(period: &Period, rows: &mut Vec<Ohlc>) {
let days: i64 = match period {
Period::OneDay => return,
Period::FiveDays => 5,
Period::OneMonth => 31,
Period::ThreeMonths => 92,
Period::SixMonths => 183,
Period::OneYear => 366,
Period::TwoYears => 731,
Period::FiveYears => 1826,
};
let Some(last_date) = rows.last().map(|item| item.date) else {
return;
};
let cutoff = last_date - chrono::Duration::days(days.saturating_sub(1));
rows.retain(|item| item.date >= cutoff);
}
fn trim_close_history_to_period(period: &Period, rows: &mut Vec<ClosePoint>) {
let days: i64 = match period {
Period::OneDay => return,
Period::FiveDays => 5,
Period::OneMonth => 31,
Period::ThreeMonths => 92,
Period::SixMonths => 183,
Period::OneYear => 366,
Period::TwoYears => 731,
Period::FiveYears => 1826,
};
let Some(last_date) = rows.last().map(|item| item.date) else {
return;
};
let cutoff = last_date - chrono::Duration::days(days.saturating_sub(1));
rows.retain(|item| item.date >= cutoff);
}
fn preferred_metric(metrics: &[IndustryMetric]) -> Option<&IndustryMetric> {
metrics.iter().max_by_key(|metric| metric_rank(metric))
}
fn best_metric_value<T: Copy>(
metrics: &[IndustryMetric],
extractor: impl Fn(&IndustryMetric) -> Option<T>,
) -> Option<T> {
metrics
.iter()
.filter_map(|metric| extractor(metric).map(|value| (metric_rank(metric), value)))
.max_by_key(|(rank, _)| *rank)
.map(|(_, value)| value)
}
fn metric_rank(metric: &IndustryMetric) -> (i32, i32) {
(
metric
.year
.as_deref()
.and_then(|year| year.parse::<i32>().ok())
.unwrap_or(i32::MIN),
metric_period_priority(metric.fiscal_period_type.as_deref()),
)
}
fn metric_period_priority(period: Option<&str>) -> i32 {
match period.map(|value| value.trim()) {
Some(value) if value.eq_ignore_ascii_case("TTM") => 7,
Some(value)
if value.eq_ignore_ascii_case("ANNUAL")
|| value.eq_ignore_ascii_case("FY")
|| value.eq_ignore_ascii_case("YEAR") =>
{
6
}
Some(value) if value.eq_ignore_ascii_case("Q4") => 5,
Some(value) if value.eq_ignore_ascii_case("Q3") => 4,
Some(value) if value.eq_ignore_ascii_case("Q2") => 3,
Some(value) if value.eq_ignore_ascii_case("Q1") => 2,
Some(value) if value.eq_ignore_ascii_case("NTM") => 1,
_ => 0,
}
}
fn normalize_percentish(value: Option<f64>) -> Option<f64> {
value.and_then(|number| {
if !number.is_finite() {
None
} else if number.abs() > 1.0 {
Some(number / 100.0)
} else {
Some(number)
}
})
}
fn sanitize_current_ratio(value: Option<f64>) -> Option<f64> {
value.and_then(|number| {
if !number.is_finite() || number < 0.01 {
None
} else {
Some(number)
}
})
}
#[allow(dead_code)]
#[derive(Clone, Copy)]
pub(super) enum ResampleInterval {
Week,
Month,
}
#[allow(dead_code)]
pub(super) fn resample_history(rows: &[Ohlc], interval: ResampleInterval) -> Vec<Ohlc> {
let mut grouped: BTreeMap<(i32, u32), Ohlc> = BTreeMap::new();
for row in rows {
let key = match interval {
ResampleInterval::Week => {
let iso = row.date.iso_week();
(iso.year(), iso.week())
}
ResampleInterval::Month => (row.date.year(), row.date.month()),
};
grouped
.entry(key)
.and_modify(|existing| {
existing.high = existing.high.max(row.high);
existing.low = existing.low.min(row.low);
existing.close = row.close;
existing.volume = existing.volume.saturating_add(row.volume);
existing.date = row.date;
})
.or_insert_with(|| row.clone());
}
grouped.into_values().collect()
}
fn parse_chart_date(raw: &str) -> Option<NaiveDate> {
if let Ok(date) = chrono::DateTime::parse_from_rfc3339(raw) {
return Some(date.date_naive());
}
if let Ok(timestamp) = raw.parse::<i64>() {
return chrono::DateTime::from_timestamp(timestamp, 0).map(|dt| dt.date_naive());
}
NaiveDate::parse_from_str(raw, "%Y-%m-%d").ok()
}
fn round_price(value: f64) -> i64 {
value.round() as i64
}
fn round_u64(value: Option<f64>) -> Option<u64> {
value.and_then(|number| {
if !number.is_finite() || number.is_sign_negative() {
None
} else {
Some(number.round() as u64)
}
})
}
fn de_opt_f64_lenient<'de, D>(deserializer: D) -> Result<Option<f64>, D::Error>
where
D: Deserializer<'de>,
{
#[derive(Deserialize)]
#[serde(untagged)]
enum NumberLike {
F64(f64),
String(String),
}
let value = Option::<NumberLike>::deserialize(deserializer)?;
match value {
Some(NumberLike::F64(number)) if number.is_finite() => Ok(Some(number)),
Some(NumberLike::F64(_)) => Ok(None),
Some(NumberLike::String(raw)) => {
let trimmed = raw.trim();
if trimmed.is_empty() || trimmed.eq_ignore_ascii_case("nan") {
Ok(None)
} else {
trimmed.parse::<f64>().map(Some).map_err(D::Error::custom)
}
}
None => Ok(None),
}
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(crate) struct MsnQuote {
#[serde(default)]
symbol: Option<String>,
price: Option<f64>,
#[serde(default)]
price_change: Option<f64>,
#[serde(default)]
price_change_percent: Option<f64>,
#[serde(default)]
price_previous_close: Option<f64>,
#[serde(default, rename = "price52wHigh")]
price_52w_high: Option<f64>,
#[serde(default, rename = "price52wLow")]
price_52w_low: Option<f64>,
#[serde(default)]
accumulated_volume: Option<f64>,
#[serde(default)]
average_volume: Option<f64>,
#[serde(default)]
market_cap: Option<f64>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(crate) struct KeyRatios {
#[serde(default)]
industry_metrics: Vec<IndustryMetric>,
#[serde(default)]
company_metrics: Vec<IndustryMetric>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
struct IndustryMetric {
year: Option<String>,
fiscal_period_type: Option<String>,
#[serde(default)]
revenue_growth_rate: Option<f64>,
#[serde(default)]
earnings_growth_rate: Option<f64>,
#[serde(default, rename = "netIncomeYTDYTDGrowthRate")]
net_income_ytd_ytd_growth_rate: Option<f64>,
#[serde(default, rename = "revenueYTDYTD")]
revenue_ytd_ytd: Option<f64>,
#[serde(default)]
net_margin: Option<f64>,
#[serde(default)]
profit_margin: Option<f64>,
#[serde(default)]
roe: Option<f64>,
#[serde(default, rename = "roaTTM")]
roa_ttm: Option<f64>,
#[serde(default)]
return_on_asset_current: Option<f64>,
#[serde(default)]
debt_to_equity_ratio: Option<f64>,
#[serde(default, deserialize_with = "de_opt_f64_lenient")]
current_ratio: Option<f64>,
#[serde(default)]
price_to_earnings_ratio: Option<f64>,
#[serde(default, rename = "forwardPriceToEPS")]
forward_price_to_eps: Option<f64>,
#[serde(default)]
price_to_book_ratio: Option<f64>,
}
#[derive(Debug, Deserialize)]
pub(crate) struct MsnChart {
series: ChartSeries,
}
#[derive(Debug, Default, Deserialize)]
#[serde(rename_all = "camelCase")]
struct ChartSeries {
#[serde(default)]
time_stamps: Vec<String>,
#[serde(default)]
prices: Vec<f64>,
#[serde(default)]
open_prices: Vec<f64>,
#[serde(default)]
prices_high: Vec<f64>,
#[serde(default)]
prices_low: Vec<f64>,
#[serde(default)]
volumes: Vec<f64>,
}
impl ChartSeries {
fn has_real_ohlcv(&self) -> bool {
!self.time_stamps.is_empty()
&& self.open_prices.len() == self.time_stamps.len()
&& self.prices_high.len() == self.time_stamps.len()
&& self.prices_low.len() == self.time_stamps.len()
&& self.prices.len() == self.time_stamps.len()
&& self.volumes.len() == self.time_stamps.len()
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
struct ClosePoint {
date: NaiveDate,
close: i64,
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::{ use super::{parse_fundamentals_from_str, parse_quote_from_str};
ResampleInterval, parse_close_only_history_from_str, parse_fundamentals_from_str, use crate::api::types::Period;
parse_history_from_str, parse_quote_from_str, resample_history,
};
use crate::api::types::{Ohlc, Period};
#[test] #[test]
fn parses_quote_fixture_json() { fn parses_quote_fixture_json() {
@ -580,160 +60,4 @@ mod tests {
assert_eq!(fundamentals.earnings_growth, Some(0.121)); assert_eq!(fundamentals.earnings_growth, Some(0.121));
assert_eq!(fundamentals.market_cap, Some(1_215_200_000_000_000)); assert_eq!(fundamentals.market_cap, Some(1_215_200_000_000_000));
} }
#[test]
fn parses_history_fixture_json() {
let raw = std::fs::read_to_string("tests/fixtures/msn_chart_bbca_3mo.json")
.expect("history fixture exists");
let history = parse_history_from_str(&Period::ThreeMonths, &raw).expect("history parsed");
assert_eq!(history.len(), 6);
assert_eq!(history[0].date.to_string(), "2025-01-06");
assert_eq!(history[0].open, 9800);
assert_eq!(history[0].close, 9875);
assert_eq!(history[5].close, 9940);
}
#[test]
fn rejects_close_only_chart_series_for_public_history() {
let raw = r#"[
{
"series": {
"prices": [7100.0, 7200.0],
"timeStamps": ["2026-03-03T17:00:00Z", "2026-03-04T17:00:00Z"]
}
}
]"#;
let err =
parse_history_from_str(&Period::ThreeMonths, raw).expect_err("history should fail");
assert_eq!(
err.to_string(),
"parse error: msn does not expose real OHLC/volume for this history range"
);
}
#[test]
fn parses_close_only_series_for_internal_use() {
let raw = r#"[
{
"series": {
"prices": [7100.0, 7200.0],
"timeStamps": ["2026-03-03T17:00:00Z", "2026-03-04T17:00:00Z"]
}
}
]"#;
let history =
parse_close_only_history_from_str(&Period::ThreeMonths, raw).expect("history parsed");
assert_eq!(history.len(), 2);
assert_eq!(history[0].close, 7100);
assert_eq!(history[1].close, 7200);
}
#[test]
fn resamples_history_to_weekly_bars() {
let rows = vec![
Ohlc {
date: chrono::NaiveDate::from_ymd_opt(2025, 1, 6).expect("date"),
open: 100,
high: 110,
low: 90,
close: 105,
volume: 10,
},
Ohlc {
date: chrono::NaiveDate::from_ymd_opt(2025, 1, 7).expect("date"),
open: 106,
high: 111,
low: 101,
close: 109,
volume: 11,
},
Ohlc {
date: chrono::NaiveDate::from_ymd_opt(2025, 1, 13).expect("date"),
open: 110,
high: 115,
low: 108,
close: 114,
volume: 12,
},
];
let weekly = resample_history(&rows, ResampleInterval::Week);
assert_eq!(weekly.len(), 2);
assert_eq!(weekly[0].open, 100);
assert_eq!(weekly[0].close, 109);
assert_eq!(weekly[0].volume, 21);
assert_eq!(weekly[1].close, 114);
}
#[test]
fn normalizes_live_style_percent_metrics() {
let raw = r#"[
{
"industryMetrics": [
{
"year": "2025",
"fiscalPeriodType": "Q1",
"revenueGrowthRate": 9.584679119559473,
"earningsGrowthRate": 28.793562408178182,
"netMargin": 35.05868669243578,
"roe": 16.27117054525313,
"returnOnAssetCurrent": 2.5707368150889867,
"debtToEquityRatio": 32.80253090283387,
"currentRatio": 9.38775908812586E-06,
"priceToEarningsRatio": 21.331183408517173,
"priceToBookRatio": 3.0625539678152234
},
{
"year": "2025",
"fiscalPeriodType": "TTM",
"revenueYTDYTD": 0.0481563350951302,
"netIncomeYTDYTDGrowthRate": 0.0492553610240516,
"profitMargin": 0.504190105842766,
"roe": 0.211493,
"roaTTM": 3.7919,
"priceToEarningsRatio": 17.296683642049683,
"priceToSalesRatio": 7.6652108104296985,
"priceToBookRatio": 3.107795874896335
},
{
"year": "2025",
"fiscalPeriodType": "NTM",
"forwardPriceToEPS": 14.723
}
],
"companyMetrics": [
{
"year": "2025",
"fiscalPeriodType": "TTM",
"revenueYTDYTD": 0.0481563350951302,
"netIncomeYTDYTDGrowthRate": 0.0492553610240516,
"profitMargin": 0.504190105842766,
"roe": 0.211493,
"roaTTM": 3.7919,
"priceToEarningsRatio": 17.296683642049683,
"priceToBookRatio": 3.107795874896335
},
{
"year": "2025",
"fiscalPeriodType": "NTM",
"forwardPriceToEPS": 14.723
}
]
}
]"#;
let quote_raw = r#"[{"symbol":"BBCA","marketCap":866500400000000.0}]"#;
let fundamentals =
parse_fundamentals_from_str(raw, Some(quote_raw)).expect("fundamentals parsed");
assert_eq!(fundamentals.trailing_pe, Some(17.296683642049683));
assert_eq!(fundamentals.forward_pe, Some(14.723));
assert_eq!(fundamentals.price_to_book, Some(3.107795874896335));
assert_eq!(fundamentals.return_on_equity, Some(0.211493));
assert_eq!(fundamentals.profit_margins, Some(0.504190105842766));
assert_eq!(fundamentals.return_on_assets, Some(0.037919));
assert_eq!(fundamentals.revenue_growth, Some(0.0481563350951302));
assert_eq!(fundamentals.earnings_growth, Some(0.0492553610240516));
assert_eq!(fundamentals.debt_to_equity, None);
assert_eq!(fundamentals.current_ratio, None);
}
} }

290
src/api/msn/raw_types.rs Normal file
View file

@ -0,0 +1,290 @@
use std::collections::HashMap;
use serde::de::Error as _;
use serde::{Deserialize, Deserializer, Serialize};
#[allow(dead_code)]
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(crate) struct MsnQuote {
#[serde(default)]
pub(crate) symbol: Option<String>,
#[serde(default)]
pub(crate) short_name: Option<String>,
pub(crate) price: Option<f64>,
#[serde(default)]
pub(crate) price_change: Option<f64>,
#[serde(default)]
pub(crate) price_change_percent: Option<f64>,
#[serde(default)]
pub(crate) price_previous_close: Option<f64>,
#[serde(default, rename = "price52wHigh")]
pub(crate) price_52w_high: Option<f64>,
#[serde(default, rename = "price52wLow")]
pub(crate) price_52w_low: Option<f64>,
#[serde(default)]
pub(crate) accumulated_volume: Option<f64>,
#[serde(default)]
pub(crate) average_volume: Option<f64>,
#[serde(default)]
pub(crate) market_cap: Option<f64>,
#[serde(default)]
pub(crate) return_ytd: Option<f64>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(crate) struct KeyRatios {
#[serde(default)]
pub(crate) industry_metrics: Vec<IndustryMetric>,
#[serde(default)]
pub(crate) company_metrics: Vec<IndustryMetric>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(crate) struct IndustryMetric {
pub(crate) year: Option<String>,
pub(crate) fiscal_period_type: Option<String>,
#[serde(default)]
pub(crate) revenue_growth_rate: Option<f64>,
#[serde(default)]
pub(crate) earnings_growth_rate: Option<f64>,
#[serde(default, rename = "netIncomeYTDYTDGrowthRate")]
pub(crate) net_income_ytd_ytd_growth_rate: Option<f64>,
#[serde(default, rename = "revenueYTDYTD")]
pub(crate) revenue_ytd_ytd: Option<f64>,
#[serde(default)]
pub(crate) net_margin: Option<f64>,
#[serde(default)]
pub(crate) profit_margin: Option<f64>,
#[serde(default)]
pub(crate) roe: Option<f64>,
#[serde(default, rename = "roaTTM")]
pub(crate) roa_ttm: Option<f64>,
#[serde(default)]
pub(crate) return_on_asset_current: Option<f64>,
#[serde(default)]
pub(crate) debt_to_equity_ratio: Option<f64>,
#[serde(default, deserialize_with = "de_opt_f64_lenient")]
pub(crate) current_ratio: Option<f64>,
#[serde(default)]
pub(crate) price_to_earnings_ratio: Option<f64>,
#[serde(default, rename = "forwardPriceToEPS")]
pub(crate) forward_price_to_eps: Option<f64>,
#[serde(default)]
pub(crate) price_to_book_ratio: Option<f64>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct RawEquity {
pub(super) id: Option<String>,
pub(super) symbol: Option<String>,
pub(super) short_name: Option<String>,
pub(super) long_name: Option<String>,
pub(super) description: Option<String>,
pub(super) sector: Option<String>,
pub(super) industry: Option<String>,
pub(super) website: Option<String>,
pub(super) full_time_employees: Option<i64>,
pub(super) address: Option<String>,
pub(super) city: Option<String>,
pub(super) country: Option<String>,
pub(super) phone: Option<String>,
pub(super) officers: Option<Vec<RawOfficer>>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct RawOfficer {
pub(super) name: Option<String>,
pub(super) title: Option<String>,
pub(super) age: Option<i32>,
pub(super) year_born: Option<i32>,
pub(super) total_pay: Option<i64>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct RawFinancialStatement {
pub(super) underlying_instrument: Option<RawInstrumentInfo>,
pub(super) balance_sheets: Option<RawStatementSection>,
pub(super) cash_flow: Option<RawStatementSection>,
#[serde(rename = "incomeStatement")]
pub(super) income_statements: Option<RawStatementSection>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct RawInstrumentInfo {
pub(super) instrument_id: Option<String>,
pub(super) display_name: Option<String>,
pub(super) short_name: Option<String>,
pub(super) symbol: Option<String>,
}
#[allow(dead_code)]
#[derive(Debug, Deserialize)]
pub(super) struct RawStatementSection {
#[serde(flatten)]
pub(super) data: HashMap<String, serde_json::Value>,
pub(super) currency: Option<String>,
pub(super) source: Option<String>,
#[serde(rename = "sourceDate")]
pub(super) source_date: Option<String>,
#[serde(rename = "reportDate")]
pub(super) report_date: Option<String>,
#[serde(rename = "endDate")]
pub(super) end_date: Option<String>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "PascalCase")]
pub(super) struct RawEarningsResponse {
pub(super) eps_last_year: Option<f64>,
pub(super) revenue_last_year: Option<f64>,
pub(super) forecast: Option<RawEarningsBucket>,
pub(super) history: Option<RawEarningsBucket>,
}
#[derive(Debug, Deserialize)]
pub(super) struct RawEarningsBucket {
pub(super) annual: Option<HashMap<String, RawEarningsData>>,
pub(super) quarterly: Option<HashMap<String, RawEarningsData>>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "PascalCase")]
pub(super) struct RawEarningsData {
pub(super) eps_actual: Option<f64>,
pub(super) eps_surprise: Option<f64>,
pub(super) eps_surprise_percent: Option<f64>,
pub(super) eps_forecast: Option<f64>,
pub(super) revenue_actual: Option<f64>,
pub(super) revenue_surprise: Option<f64>,
pub(super) revenue_forecast: Option<f64>,
pub(super) earning_release_date: Option<String>,
pub(super) ciq_fiscal_period_type: Option<String>,
}
#[allow(dead_code)]
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct RawSentiment {
pub(super) symbol: Option<String>,
pub(super) display_name: Option<String>,
pub(super) sentiment_statistics: Option<Vec<RawSentimentStat>>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct RawSentimentStat {
pub(super) time_range_name: Option<String>,
pub(super) bullish: Option<i32>,
pub(super) bearish: Option<i32>,
pub(super) neutral: Option<i32>,
}
// Actual MSN insights API response: array of insight containers, each holding
// individual insight items grouped by category (Valuation, Risk, etc.)
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct RawInsight {
pub(super) instrument_id: Option<String>,
pub(super) display_name: Option<String>,
pub(super) insights: Option<Vec<RawInsightItem>>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct RawInsightItem {
pub(super) insight_name: Option<String>,
pub(super) category: Option<String>,
pub(super) insight_statement: Option<String>,
}
#[derive(Debug, Deserialize)]
pub(super) struct RawNewsFeed {
pub(super) value: Option<Vec<RawNewsItem>>,
#[serde(rename = "subCards")]
pub(super) sub_cards: Option<Vec<RawNewsItem>>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct RawNewsItem {
pub(super) id: Option<String>,
pub(super) title: Option<String>,
pub(super) url: Option<String>,
#[serde(rename = "abstract")]
pub(super) description: Option<String>,
pub(super) provider: Option<RawNewsProvider>,
pub(super) published_date_time: Option<String>,
pub(super) read_time_min: Option<i32>,
}
#[derive(Debug, Deserialize)]
pub(super) struct RawNewsProvider {
pub(super) name: Option<String>,
}
#[derive(Debug, Serialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct ScreenerRequest {
pub(super) filter: Vec<ScreenerFilter>,
pub(super) order: ScreenerOrder,
pub(super) return_value_type: Vec<String>,
pub(super) screener_type: String,
pub(super) limit: usize,
pub(super) page_index: usize,
}
#[derive(Debug, Serialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct ScreenerFilter {
pub(super) key: String,
pub(super) key_group: String,
pub(super) is_range: bool,
}
#[derive(Debug, Serialize)]
pub(super) struct ScreenerOrder {
pub(super) key: String,
pub(super) dir: String,
}
#[allow(dead_code)]
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct RawScreenerResponse {
pub(super) count: Option<i32>,
pub(super) quote: Option<Vec<MsnQuote>>,
}
fn de_opt_f64_lenient<'de, D>(deserializer: D) -> Result<Option<f64>, D::Error>
where
D: Deserializer<'de>,
{
#[derive(Deserialize)]
#[serde(untagged)]
enum NumberLike {
F64(f64),
String(String),
}
let value = Option::<NumberLike>::deserialize(deserializer)?;
match value {
Some(NumberLike::F64(number)) if number.is_finite() => Ok(Some(number)),
Some(NumberLike::F64(_)) => Ok(None),
Some(NumberLike::String(raw)) => {
let trimmed = raw.trim();
if trimmed.is_empty() || trimmed.eq_ignore_ascii_case("nan") {
Ok(None)
} else {
trimmed.parse::<f64>().map(Some).map_err(D::Error::custom)
}
}
None => Ok(None),
}
}

View file

@ -82,6 +82,125 @@ pub struct Fundamentals {
pub market_cap: Option<u64>, pub market_cap: Option<u64>,
} }
pub type Bar = Ohlc;
// Forward-looking types for planned MSN endpoints — used once capability traits are wired to CLI.
#[allow(dead_code)]
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CompanyProfile {
pub id: String,
pub symbol: String,
pub short_name: String,
pub long_name: String,
pub description: String,
pub sector: String,
pub industry: String,
pub website: String,
pub employees: i64,
pub address: String,
pub city: String,
pub country: String,
pub phone: String,
pub officers: Vec<Officer>,
}
#[allow(dead_code)]
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Officer {
pub name: String,
pub title: String,
pub age: Option<i32>,
pub year_born: Option<i32>,
pub total_pay: Option<i64>,
}
#[allow(dead_code)]
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct FinancialStatements {
pub instrument: InstrumentInfo,
pub balance_sheet: Option<StatementSection>,
pub cash_flow: Option<StatementSection>,
pub income_statement: Option<StatementSection>,
}
#[allow(dead_code)]
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct InstrumentInfo {
pub id: String,
pub symbol: String,
pub name: String,
}
#[allow(dead_code)]
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct StatementSection {
pub values: std::collections::HashMap<String, f64>,
pub currency: String,
pub report_date: String,
pub end_date: String,
}
#[allow(dead_code)]
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct EarningsReport {
pub eps_last_year: f64,
pub revenue_last_year: f64,
pub forecast: Vec<EarningsData>,
pub history: Vec<EarningsData>,
}
#[allow(dead_code)]
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct EarningsData {
pub eps_actual: Option<f64>,
pub eps_forecast: Option<f64>,
pub eps_surprise: Option<f64>,
pub eps_surprise_pct: Option<f64>,
pub revenue_actual: Option<f64>,
pub revenue_forecast: Option<f64>,
pub revenue_surprise: Option<f64>,
pub earning_release_date: Option<String>,
pub period_type: String,
}
#[allow(dead_code)]
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SentimentData {
pub symbol: String,
pub statistics: Vec<SentimentPeriod>,
}
#[allow(dead_code)]
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SentimentPeriod {
pub time_range: String,
pub bullish: i32,
pub bearish: i32,
pub neutral: i32,
}
#[allow(dead_code)]
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct InsightData {
pub id: String,
pub summary: String,
pub highlights: Vec<String>,
pub risks: Vec<String>,
pub last_updated: String,
}
#[allow(dead_code)]
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct NewsItem {
pub id: String,
pub title: String,
pub url: String,
pub description: String,
pub provider: String,
pub published_at: String,
pub read_time_min: Option<i32>,
}
#[derive(Debug, Deserialize)] #[derive(Debug, Deserialize)]
#[serde(untagged)] #[serde(untagged)]
enum NumberLike { enum NumberLike {

View file

@ -6,7 +6,7 @@ use std::time::Duration;
use crate::api::types::{Interval, Period}; use crate::api::types::{Interval, Period};
use crate::error::IdxError; use crate::error::IdxError;
use super::parse::{ChartResponse, QuoteSummaryResponse}; use super::raw_types::{ChartResponse, QuoteSummaryResponse};
const USER_AGENT: &str = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36"; const USER_AGENT: &str = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36";
const BASE_URL: &str = "https://query2.finance.yahoo.com"; const BASE_URL: &str = "https://query2.finance.yahoo.com";
@ -42,7 +42,7 @@ impl YahooClient {
fn quote_summary_url(symbol: &str, crumb: &str) -> String { fn quote_summary_url(symbol: &str, crumb: &str) -> String {
format!( format!(
"{BASE_URL}/v10/finance/quoteSummary/{symbol}?modules=defaultKeyStatistics,financialData,incomeStatementHistory&crumb={crumb}" "{BASE_URL}/v10/finance/quoteSummary/{symbol}?modules=summaryDetail,defaultKeyStatistics,financialData,assetProfile,incomeStatementHistory&crumb={crumb}"
) )
} }
@ -262,8 +262,15 @@ impl YahooClient {
) -> Result<QuoteSummaryResponse, IdxError> { ) -> Result<QuoteSummaryResponse, IdxError> {
for auth_attempt in 0..2 { for auth_attempt in 0..2 {
let crumb = self.get_or_init_crumb()?; let crumb = self.get_or_init_crumb()?;
let cookie_header = let cookie_header = match Self::cookie_header_from_jar(&Self::cookie_jar_path()) {
Self::cookie_header_from_jar(&Self::cookie_jar_path()).unwrap_or_default(); Ok(header) => header,
Err(err) => {
eprintln!("warning: failed to parse Yahoo cookie jar: {err}");
return Err(IdxError::AuthError(format!(
"failed to parse Yahoo cookies: {err}"
)));
}
};
let url = Self::quote_summary_url(symbol, &crumb); let url = Self::quote_summary_url(symbol, &crumb);
let mut wait = Duration::from_millis(250); let mut wait = Duration::from_millis(250);

209
src/api/yahoo/map.rs Normal file
View file

@ -0,0 +1,209 @@
use crate::api::types::{Fundamentals, Ohlc, Quote};
use crate::error::IdxError;
use super::raw_types::{ChartError, ChartResponse, QuoteSummaryResponse};
pub(super) fn parse_quote(symbol: &str, chart: &ChartResponse) -> Result<Quote, IdxError> {
if let Some(err) = chart.chart.error.as_ref() {
return Err(map_yahoo_error(symbol, "chart", err));
}
let result = chart
.chart
.result
.as_ref()
.and_then(|r| r.first())
.ok_or(IdxError::ProviderUnavailable)?;
let meta = result.meta.as_ref().ok_or(IdxError::ProviderUnavailable)?;
let raw_price = meta
.regular_market_price
.ok_or_else(|| IdxError::SymbolNotFound(symbol.to_string()))?;
let raw_prev_close = meta.previous_close.or(meta.chart_previous_close);
let price = round_price(raw_price);
let prev_close = raw_prev_close.map(round_price);
let change = prev_close.map_or(0, |p| price - p);
let change_pct = raw_prev_close.map_or(0.0, |p| {
if p != 0.0 {
((raw_price - p) / p) * 100.0
} else {
0.0
}
});
let (week52_position, range_signal) = match (meta.fifty_two_week_low, meta.fifty_two_week_high)
{
(Some(low), Some(high)) if high > low => {
let pos = (raw_price - low) / (high - low);
let signal = if pos > 0.66 {
"upper"
} else if pos < 0.33 {
"lower"
} else {
"middle"
};
(Some(pos), Some(signal.to_string()))
}
_ => (None, None),
};
Ok(Quote {
symbol: meta.symbol.clone().unwrap_or_else(|| symbol.to_string()),
price,
change,
change_pct,
volume: meta.regular_market_volume.unwrap_or(0),
market_cap: meta.market_cap,
week52_high: meta.fifty_two_week_high.map(round_price),
week52_low: meta.fifty_two_week_low.map(round_price),
week52_position,
range_signal,
prev_close,
avg_volume: meta.average_daily_volume_3month,
})
}
pub(super) fn parse_history(
symbol: &str,
chart: &ChartResponse,
) -> Result<(Vec<Ohlc>, usize), IdxError> {
if let Some(err) = chart.chart.error.as_ref() {
return Err(map_yahoo_error(symbol, "chart", err));
}
let result = chart
.chart
.result
.as_ref()
.and_then(|r| r.first())
.ok_or(IdxError::ProviderUnavailable)?;
let timestamps = result
.timestamp
.as_ref()
.ok_or(IdxError::ProviderUnavailable)?;
let quote = result
.indicators
.as_ref()
.and_then(|i| i.quote.as_ref())
.and_then(|q| q.first())
.ok_or(IdxError::ProviderUnavailable)?;
let mut out = Vec::new();
let mut dropped = 0usize;
for (i, ts) in timestamps.iter().enumerate() {
let open = quote
.open
.as_ref()
.and_then(|v| v.get(i).copied().flatten())
.map(round_price);
let high = quote
.high
.as_ref()
.and_then(|v| v.get(i).copied().flatten())
.map(round_price);
let low = quote
.low
.as_ref()
.and_then(|v| v.get(i).copied().flatten())
.map(round_price);
let close = quote
.close
.as_ref()
.and_then(|v| v.get(i).copied().flatten())
.map(round_price);
let volume = quote
.volume
.as_ref()
.and_then(|v| v.get(i).copied().flatten());
if let (Some(open), Some(high), Some(low), Some(close), Some(volume)) =
(open, high, low, close, volume)
&& let Some(dt) = chrono::DateTime::from_timestamp(*ts, 0)
{
out.push(Ohlc {
date: dt.date_naive(),
open,
high,
low,
close,
volume,
});
} else {
dropped += 1;
}
}
Ok((out, dropped))
}
pub(super) fn parse_fundamentals(
symbol: &str,
quote_summary: &QuoteSummaryResponse,
) -> Result<Fundamentals, IdxError> {
if let Some(err) = quote_summary.quote_summary.error.as_ref() {
return Err(map_yahoo_error(symbol, "quoteSummary", err));
}
let result = quote_summary
.quote_summary
.result
.as_ref()
.and_then(|results| results.first())
.ok_or(IdxError::ProviderUnavailable)?;
let stats = result.default_key_statistics.as_ref();
let fin = result.financial_data.as_ref();
let summary = result.summary_detail.as_ref();
Ok(Fundamentals {
trailing_pe: stats
.and_then(|s| s.trailing_pe.as_ref().and_then(|v| v.raw))
.or_else(|| fin.and_then(|f| f.trailing_pe.as_ref().and_then(|v| v.raw)))
.or_else(|| summary.and_then(|s| s.trailing_pe.as_ref().and_then(|v| v.raw))),
forward_pe: stats
.and_then(|s| s.forward_pe.as_ref().and_then(|v| v.raw))
.or_else(|| fin.and_then(|f| f.forward_pe.as_ref().and_then(|v| v.raw)))
.or_else(|| summary.and_then(|s| s.forward_pe.as_ref().and_then(|v| v.raw))),
price_to_book: stats
.and_then(|s| s.price_to_book.as_ref().and_then(|v| v.raw))
.or_else(|| fin.and_then(|f| f.price_to_book.as_ref().and_then(|v| v.raw)))
.or_else(|| summary.and_then(|s| s.price_to_book.as_ref().and_then(|v| v.raw))),
return_on_equity: fin.and_then(|f| f.return_on_equity.as_ref().and_then(|v| v.raw)),
profit_margins: fin.and_then(|f| f.profit_margins.as_ref().and_then(|v| v.raw)),
return_on_assets: fin.and_then(|f| f.return_on_assets.as_ref().and_then(|v| v.raw)),
revenue_growth: fin.and_then(|f| f.revenue_growth.as_ref().and_then(|v| v.raw)),
earnings_growth: stats
.and_then(|s| s.earnings_growth.as_ref().and_then(|v| v.raw))
.or_else(|| fin.and_then(|f| f.earnings_growth.as_ref().and_then(|v| v.raw))),
debt_to_equity: fin.and_then(|f| f.debt_to_equity.as_ref().and_then(|v| v.raw)),
current_ratio: fin.and_then(|f| f.current_ratio.as_ref().and_then(|v| v.raw)),
enterprise_value: stats
.and_then(|s| s.enterprise_value.as_ref().and_then(|v| v.raw))
.or_else(|| fin.and_then(|f| f.enterprise_value.as_ref().and_then(|v| v.raw))),
ebitda: fin
.and_then(|f| f.ebitda.as_ref().and_then(|v| v.raw))
.or_else(|| stats.and_then(|s| s.ebitda.as_ref().and_then(|v| v.raw))),
market_cap: fin
.and_then(|f| f.market_cap.as_ref().and_then(|v| v.raw))
.or_else(|| stats.and_then(|s| s.market_cap.as_ref().and_then(|v| v.raw)))
.or_else(|| {
summary
.and_then(|s| s.market_cap.as_ref().and_then(|v| v.raw))
.map(|n| n.round() as u64)
}),
})
}
fn round_price(value: f64) -> i64 {
value.round() as i64
}
pub(super) fn map_yahoo_error(symbol: &str, endpoint: &str, err: &ChartError) -> IdxError {
if err.code.eq_ignore_ascii_case("Not Found") {
return IdxError::SymbolNotFound(symbol.to_string());
}
IdxError::Http(format!(
"yahoo {endpoint} error {}: {}",
err.code, err.description
))
}

View file

@ -1,12 +1,15 @@
mod client; mod client;
mod map;
mod parse; mod parse;
mod raw_types;
use crate::api::MarketDataProvider; use crate::api::types::{Bar, Fundamentals, Interval, Period, Quote};
use crate::api::types::{Fundamentals, Interval, Ohlc, Period, Quote}; use crate::api::{FundamentalsProvider, HistoryProvider, QuoteProvider};
use crate::error::IdxError; use crate::error::IdxError;
use client::YahooClient; use client::YahooClient;
use parse::{parse_fundamentals, parse_history_with_verbose, parse_quote}; use map::{parse_fundamentals, parse_quote};
use parse::parse_history_with_verbose;
pub(crate) use parse::{parse_fundamentals_from_str, parse_history_from_str, parse_quote_from_str}; pub(crate) use parse::{parse_fundamentals_from_str, parse_history_from_str, parse_quote_from_str};
@ -24,26 +27,30 @@ impl YahooProvider {
} }
} }
impl MarketDataProvider for YahooProvider { impl QuoteProvider for YahooProvider {
fn quote(&self, symbol: &str) -> Result<Quote, IdxError> { fn quote(&self, symbol: &str) -> Result<Quote, IdxError> {
let chart = self let chart = self
.client .client
.fetch_chart(symbol, &Period::OneDay, &Interval::Day)?; .fetch_chart(symbol, &Period::OneDay, &Interval::Day)?;
parse_quote(symbol, &chart) parse_quote(symbol, &chart)
} }
}
impl FundamentalsProvider for YahooProvider {
fn fundamentals(&self, symbol: &str) -> Result<Fundamentals, IdxError> { fn fundamentals(&self, symbol: &str) -> Result<Fundamentals, IdxError> {
let quote_summary = self.client.fetch_quote_summary(symbol)?; let quote_summary = self.client.fetch_quote_summary(symbol)?;
parse_fundamentals(symbol, &quote_summary) parse_fundamentals(symbol, &quote_summary)
} }
}
impl HistoryProvider for YahooProvider {
fn history( fn history(
&self, &self,
symbol: &str, symbol: &str,
period: &Period, period: &Period,
interval: &Interval, interval: &Interval,
) -> Result<Vec<Ohlc>, IdxError> { ) -> Result<Vec<Bar>, IdxError> {
let chart = self.client.fetch_chart(symbol, period, interval)?; let chart = self.client.fetch_chart(symbol, period, interval)?;
parse_history_with_verbose(&chart, self.verbose) parse_history_with_verbose(symbol, &chart, self.verbose)
} }
} }

View file

@ -1,83 +1,19 @@
use std::collections::HashMap;
use serde::Deserialize;
use crate::api::types::{Fundamentals, Ohlc, Quote}; use crate::api::types::{Fundamentals, Ohlc, Quote};
use crate::error::IdxError; use crate::error::IdxError;
use super::map::{parse_fundamentals, parse_history, parse_quote};
use super::raw_types::{ChartResponse, QuoteSummaryResponse};
pub(crate) fn parse_quote_from_str(symbol: &str, raw: &str) -> Result<Quote, IdxError> { pub(crate) fn parse_quote_from_str(symbol: &str, raw: &str) -> Result<Quote, IdxError> {
let chart: ChartResponse = let chart: ChartResponse =
serde_json::from_str(raw).map_err(|e| IdxError::ParseError(e.to_string()))?; serde_json::from_str(raw).map_err(|e| IdxError::ParseError(e.to_string()))?;
if let Some(err) = chart.chart.error.as_ref() {
return Err(map_yahoo_error(symbol, "chart", err));
}
parse_quote(symbol, &chart) parse_quote(symbol, &chart)
} }
pub(super) fn parse_quote(symbol: &str, chart: &ChartResponse) -> Result<Quote, IdxError> { pub(crate) fn parse_history_from_str(symbol: &str, raw: &str) -> Result<Vec<Ohlc>, IdxError> {
if let Some(err) = chart.chart.error.as_ref() {
return Err(map_yahoo_error(symbol, "chart", err));
}
let result = chart
.chart
.result
.as_ref()
.and_then(|r| r.first())
.ok_or(IdxError::ProviderUnavailable)?;
let meta = result.meta.as_ref().ok_or(IdxError::ProviderUnavailable)?;
let raw_price = meta
.regular_market_price
.ok_or_else(|| IdxError::SymbolNotFound(symbol.to_string()))?;
let raw_prev_close = meta.previous_close.or(meta.chart_previous_close);
let price = round_price(raw_price);
let prev_close = raw_prev_close.map(round_price);
let change = prev_close.map_or(0, |p| price - p);
let change_pct = raw_prev_close.map_or(0.0, |p| {
if p != 0.0 {
((raw_price - p) / p) * 100.0
} else {
0.0
}
});
let (week52_position, range_signal) = match (meta.fifty_two_week_low, meta.fifty_two_week_high)
{
(Some(low), Some(high)) if high > low => {
let pos = (raw_price - low) / (high - low);
let signal = if pos > 0.66 {
"upper"
} else if pos < 0.33 {
"lower"
} else {
"middle"
};
(Some(pos), Some(signal.to_string()))
}
_ => (None, None),
};
Ok(Quote {
symbol: meta.symbol.clone().unwrap_or_else(|| symbol.to_string()),
price,
change,
change_pct,
volume: meta.regular_market_volume.unwrap_or(0),
market_cap: meta.market_cap,
week52_high: meta.fifty_two_week_high.map(round_price),
week52_low: meta.fifty_two_week_low.map(round_price),
week52_position,
range_signal,
prev_close,
avg_volume: meta.average_daily_volume_3month,
})
}
pub(crate) fn parse_history_from_str(raw: &str) -> Result<Vec<Ohlc>, IdxError> {
let chart: ChartResponse = let chart: ChartResponse =
serde_json::from_str(raw).map_err(|e| IdxError::ParseError(e.to_string()))?; serde_json::from_str(raw).map_err(|e| IdxError::ParseError(e.to_string()))?;
parse_history_with_verbose(&chart, false) parse_history_with_verbose(symbol, &chart, false)
} }
pub(crate) fn parse_fundamentals_from_str( pub(crate) fn parse_fundamentals_from_str(
@ -86,327 +22,21 @@ pub(crate) fn parse_fundamentals_from_str(
) -> Result<Fundamentals, IdxError> { ) -> Result<Fundamentals, IdxError> {
let quote_summary: QuoteSummaryResponse = let quote_summary: QuoteSummaryResponse =
serde_json::from_str(raw).map_err(|e| IdxError::ParseError(e.to_string()))?; serde_json::from_str(raw).map_err(|e| IdxError::ParseError(e.to_string()))?;
if let Some(err) = quote_summary.quote_summary.error.as_ref() {
return Err(map_yahoo_error(symbol, "quoteSummary", err));
}
parse_fundamentals(symbol, &quote_summary) parse_fundamentals(symbol, &quote_summary)
} }
pub(super) fn parse_history_with_verbose( pub(super) fn parse_history_with_verbose(
symbol: &str,
chart: &ChartResponse, chart: &ChartResponse,
verbose: bool, verbose: bool,
) -> Result<Vec<Ohlc>, IdxError> { ) -> Result<Vec<Ohlc>, IdxError> {
if let Some(err) = chart.chart.error.as_ref() { let (history, dropped) = parse_history(symbol, chart)?;
return Err(map_yahoo_error("unknown", "chart", err));
}
let result = chart
.chart
.result
.as_ref()
.and_then(|r| r.first())
.ok_or(IdxError::ProviderUnavailable)?;
let timestamps = result
.timestamp
.as_ref()
.ok_or(IdxError::ProviderUnavailable)?;
let quote = result
.indicators
.as_ref()
.and_then(|i| i.quote.as_ref())
.and_then(|q| q.first())
.ok_or(IdxError::ProviderUnavailable)?;
let mut out = Vec::new();
let mut dropped = 0usize;
for (i, ts) in timestamps.iter().enumerate() {
let open = quote
.open
.as_ref()
.and_then(|v| v.get(i).copied().flatten())
.map(round_price);
let high = quote
.high
.as_ref()
.and_then(|v| v.get(i).copied().flatten())
.map(round_price);
let low = quote
.low
.as_ref()
.and_then(|v| v.get(i).copied().flatten())
.map(round_price);
let close = quote
.close
.as_ref()
.and_then(|v| v.get(i).copied().flatten())
.map(round_price);
let volume = quote
.volume
.as_ref()
.and_then(|v| v.get(i).copied().flatten());
if let (Some(open), Some(high), Some(low), Some(close), Some(volume)) =
(open, high, low, close, volume)
&& let Some(dt) = chrono::DateTime::from_timestamp(*ts, 0)
{
out.push(Ohlc {
date: dt.date_naive(),
open,
high,
low,
close,
volume,
});
} else {
dropped += 1;
}
}
if dropped > 0 && verbose { if dropped > 0 && verbose {
eprintln!( eprintln!(
"warning: dropped {dropped} OHLC row(s) from Yahoo response due to missing fields" "warning: dropped {dropped} OHLC row(s) from Yahoo response due to missing fields"
); );
} }
Ok(history)
Ok(out)
}
pub(super) fn parse_fundamentals(
symbol: &str,
quote_summary: &QuoteSummaryResponse,
) -> Result<Fundamentals, IdxError> {
if let Some(err) = quote_summary.quote_summary.error.as_ref() {
return Err(map_yahoo_error(symbol, "quoteSummary", err));
}
let result = quote_summary
.quote_summary
.result
.as_ref()
.and_then(|results| results.first())
.ok_or(IdxError::ProviderUnavailable)?;
Ok(Fundamentals {
trailing_pe: result
.default_key_statistics
.get_f64("trailingPE")
.or_else(|| result.financial_data.get_f64("trailingPE")),
forward_pe: result
.default_key_statistics
.get_f64("forwardPE")
.or_else(|| result.financial_data.get_f64("forwardPE")),
price_to_book: result
.default_key_statistics
.get_f64("priceToBook")
.or_else(|| result.financial_data.get_f64("priceToBook")),
return_on_equity: result.financial_data.get_f64("returnOnEquity"),
profit_margins: result.financial_data.get_f64("profitMargins"),
return_on_assets: result.financial_data.get_f64("returnOnAssets"),
revenue_growth: result.financial_data.get_f64("revenueGrowth"),
earnings_growth: result
.default_key_statistics
.get_f64("earningsGrowth")
.or_else(|| result.financial_data.get_f64("earningsGrowth")),
debt_to_equity: result.financial_data.get_f64("debtToEquity"),
current_ratio: result.financial_data.get_f64("currentRatio"),
enterprise_value: result
.default_key_statistics
.get_i64("enterpriseValue")
.or_else(|| result.financial_data.get_i64("enterpriseValue")),
ebitda: result
.financial_data
.get_i64("ebitda")
.or_else(|| result.default_key_statistics.get_i64("ebitda")),
market_cap: result
.financial_data
.get_u64("marketCap")
.or_else(|| result.default_key_statistics.get_u64("marketCap")),
})
}
fn round_price(value: f64) -> i64 {
value.round() as i64
}
pub(super) fn map_yahoo_error(symbol: &str, endpoint: &str, err: &ChartError) -> IdxError {
if err.code.eq_ignore_ascii_case("Not Found") {
return IdxError::SymbolNotFound(symbol.to_string());
}
IdxError::Http(format!(
"yahoo {endpoint} error {}: {}",
err.code, err.description
))
}
#[derive(Debug, Deserialize)]
pub(super) struct ChartResponse {
chart: ChartRoot,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct QuoteSummaryResponse {
quote_summary: QuoteSummaryRoot,
}
#[derive(Debug, Deserialize)]
pub(super) struct QuoteSummaryRoot {
result: Option<Vec<QuoteSummaryResult>>,
error: Option<ChartError>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct QuoteSummaryResult {
#[serde(default)]
default_key_statistics: QuoteSummarySection,
#[serde(default)]
financial_data: QuoteSummarySection,
}
#[derive(Debug, Deserialize)]
pub(super) struct ChartRoot {
result: Option<Vec<ChartResult>>,
error: Option<ChartError>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct ChartError {
code: String,
description: String,
}
#[derive(Debug, Deserialize)]
pub(super) struct ChartResult {
meta: Option<ChartMeta>,
timestamp: Option<Vec<i64>>,
indicators: Option<Indicators>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
#[allow(dead_code)]
pub(super) struct ChartMeta {
symbol: Option<String>,
regular_market_price: Option<f64>,
previous_close: Option<f64>,
chart_previous_close: Option<f64>,
regular_market_volume: Option<u64>,
regular_market_day_high: Option<f64>,
regular_market_day_low: Option<f64>,
market_cap: Option<u64>,
fifty_two_week_high: Option<f64>,
fifty_two_week_low: Option<f64>,
#[serde(rename = "averageDailyVolume3Month")]
average_daily_volume_3month: Option<u64>,
}
#[derive(Debug, Deserialize)]
pub(super) struct Indicators {
quote: Option<Vec<IndicatorQuote>>,
}
#[derive(Debug, Deserialize)]
pub(super) struct IndicatorQuote {
open: Option<Vec<Option<f64>>>,
high: Option<Vec<Option<f64>>>,
low: Option<Vec<Option<f64>>>,
close: Option<Vec<Option<f64>>>,
volume: Option<Vec<Option<u64>>>,
}
type QuoteSummarySection = HashMap<String, QuoteSummaryValue>;
trait QuoteSummarySectionExt {
fn get_f64(&self, key: &str) -> Option<f64>;
fn get_i64(&self, key: &str) -> Option<i64>;
fn get_u64(&self, key: &str) -> Option<u64>;
}
impl QuoteSummarySectionExt for QuoteSummarySection {
fn get_f64(&self, key: &str) -> Option<f64> {
self.get(key).and_then(QuoteSummaryValue::as_f64)
}
fn get_i64(&self, key: &str) -> Option<i64> {
self.get(key).and_then(QuoteSummaryValue::as_i64)
}
fn get_u64(&self, key: &str) -> Option<u64> {
self.get(key).and_then(QuoteSummaryValue::as_u64)
}
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
#[allow(dead_code)]
enum QuoteSummaryValue {
Wrapped { raw: Option<YahooNumber> },
Direct(YahooNumber),
// Catch-all for empty objects {}, null, strings, booleans; return None for numeric extractions.
Unknown(serde_json::Value),
}
impl QuoteSummaryValue {
fn as_f64(&self) -> Option<f64> {
match self {
Self::Wrapped { raw } => raw.as_ref().map(YahooNumber::as_f64),
Self::Direct(value) => Some(value.as_f64()),
Self::Unknown(_) => None,
}
}
fn as_i64(&self) -> Option<i64> {
match self {
Self::Wrapped { raw } => raw.as_ref().and_then(YahooNumber::as_i64),
Self::Direct(value) => value.as_i64(),
Self::Unknown(_) => None,
}
}
fn as_u64(&self) -> Option<u64> {
match self {
Self::Wrapped { raw } => raw.as_ref().and_then(YahooNumber::as_u64),
Self::Direct(value) => value.as_u64(),
Self::Unknown(_) => None,
}
}
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum YahooNumber {
I64(i64),
U64(u64),
F64(f64),
}
impl YahooNumber {
fn as_f64(&self) -> f64 {
match self {
Self::I64(value) => *value as f64,
Self::U64(value) => *value as f64,
Self::F64(value) => *value,
}
}
fn as_i64(&self) -> Option<i64> {
match self {
Self::I64(value) => Some(*value),
Self::U64(value) => i64::try_from(*value).ok(),
Self::F64(value) => Some(value.round() as i64),
}
}
fn as_u64(&self) -> Option<u64> {
match self {
Self::I64(value) => u64::try_from(*value).ok(),
Self::U64(value) => Some(*value),
Self::F64(value) if value.is_sign_negative() => None,
Self::F64(value) => Some(value.round() as u64),
}
}
} }
#[cfg(test)] #[cfg(test)]
@ -447,7 +77,7 @@ mod tests {
let quote = parse_quote("BBCA.JK", &chart).expect("quote parsed"); let quote = parse_quote("BBCA.JK", &chart).expect("quote parsed");
assert_eq!(quote.symbol, "BBCA.JK"); assert_eq!(quote.symbol, "BBCA.JK");
assert_eq!(quote.price, 9875); assert_eq!(quote.price, 9875);
let history = parse_history_with_verbose(&chart, false).expect("history parsed"); let history = parse_history_with_verbose("BBCA.JK", &chart, false).expect("history parsed");
assert_eq!(history.len(), 2); assert_eq!(history.len(), 2);
assert_eq!(history[0].close, 9875); assert_eq!(history[0].close, 9875);
} }
@ -466,7 +96,8 @@ mod tests {
assert_eq!(quote.market_cap, Some(1_215_200_000_000_000)); assert_eq!(quote.market_cap, Some(1_215_200_000_000_000));
assert_eq!(quote.avg_volume, Some(10_000_000)); assert_eq!(quote.avg_volume, Some(10_000_000));
let history = parse_history_from_str(&history_raw).expect("fixture history parsed"); let history =
parse_history_from_str("BBCA.JK", &history_raw).expect("fixture history parsed");
assert!(!history.is_empty()); assert!(!history.is_empty());
let fundamentals = parse_fundamentals_from_str("BBCA.JK", &fundamentals_raw) let fundamentals = parse_fundamentals_from_str("BBCA.JK", &fundamentals_raw)

156
src/api/yahoo/raw_types.rs Normal file
View file

@ -0,0 +1,156 @@
// Raw serde structs for Yahoo API responses. Fields not yet consumed by map.rs are
// retained for future fundamentals expansion; suppress dead_code for forward-compat.
#![allow(dead_code)]
use serde::Deserialize;
#[derive(Debug, Deserialize)]
pub(super) struct ChartResponse {
pub(super) chart: ChartRoot,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct QuoteSummaryResponse {
pub(super) quote_summary: QuoteSummaryRoot,
}
#[derive(Debug, Deserialize)]
pub(super) struct QuoteSummaryRoot {
pub(super) result: Option<Vec<QuoteSummaryResult>>,
pub(super) error: Option<ChartError>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct QuoteSummaryResult {
#[serde(default)]
pub(super) summary_detail: Option<SummaryDetail>,
#[serde(default)]
pub(super) default_key_statistics: Option<DefaultKeyStatistics>,
#[serde(default)]
pub(super) financial_data: Option<FinancialData>,
#[serde(default)]
pub(super) asset_profile: Option<AssetProfile>,
}
#[derive(Debug, Deserialize)]
pub(super) struct ChartRoot {
pub(super) result: Option<Vec<ChartResult>>,
pub(super) error: Option<ChartError>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct ChartError {
pub(super) code: String,
pub(super) description: String,
}
#[derive(Debug, Deserialize)]
pub(super) struct ChartResult {
pub(super) meta: Option<ChartMeta>,
pub(super) timestamp: Option<Vec<i64>>,
pub(super) indicators: Option<Indicators>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
#[allow(dead_code)]
pub(super) struct ChartMeta {
pub(super) symbol: Option<String>,
pub(super) regular_market_price: Option<f64>,
pub(super) previous_close: Option<f64>,
pub(super) chart_previous_close: Option<f64>,
pub(super) regular_market_volume: Option<u64>,
pub(super) regular_market_day_high: Option<f64>,
pub(super) regular_market_day_low: Option<f64>,
pub(super) market_cap: Option<u64>,
pub(super) fifty_two_week_high: Option<f64>,
pub(super) fifty_two_week_low: Option<f64>,
#[serde(rename = "averageDailyVolume3Month")]
pub(super) average_daily_volume_3month: Option<u64>,
}
#[derive(Debug, Deserialize)]
pub(super) struct Indicators {
pub(super) quote: Option<Vec<IndicatorQuote>>,
}
#[derive(Debug, Deserialize)]
pub(super) struct IndicatorQuote {
pub(super) open: Option<Vec<Option<f64>>>,
pub(super) high: Option<Vec<Option<f64>>>,
pub(super) low: Option<Vec<Option<f64>>>,
pub(super) close: Option<Vec<Option<f64>>>,
pub(super) volume: Option<Vec<Option<u64>>>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct SummaryDetail {
#[serde(rename = "trailingPE")]
pub trailing_pe: Option<FloatValue>,
#[serde(rename = "forwardPE")]
pub forward_pe: Option<FloatValue>,
pub price_to_book: Option<FloatValue>,
pub dividend_yield: Option<FloatValue>,
pub market_cap: Option<FloatValue>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct DefaultKeyStatistics {
#[serde(rename = "trailingPE")]
pub trailing_pe: Option<FloatValue>,
#[serde(rename = "forwardPE")]
pub forward_pe: Option<FloatValue>,
pub price_to_book: Option<FloatValue>,
pub earnings_growth: Option<FloatValue>,
pub enterprise_value: Option<IntValue>,
pub ebitda: Option<IntValue>,
pub market_cap: Option<UIntValue>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct FinancialData {
#[serde(rename = "trailingPE")]
pub trailing_pe: Option<FloatValue>,
#[serde(rename = "forwardPE")]
pub forward_pe: Option<FloatValue>,
pub price_to_book: Option<FloatValue>,
pub return_on_equity: Option<FloatValue>,
pub profit_margins: Option<FloatValue>,
pub return_on_assets: Option<FloatValue>,
pub revenue_growth: Option<FloatValue>,
pub earnings_growth: Option<FloatValue>,
pub debt_to_equity: Option<FloatValue>,
pub current_ratio: Option<FloatValue>,
pub enterprise_value: Option<IntValue>,
pub ebitda: Option<IntValue>,
pub market_cap: Option<UIntValue>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct AssetProfile {
pub sector: Option<String>,
pub industry: Option<String>,
pub long_business_summary: Option<String>,
}
#[derive(Debug, Deserialize)]
pub struct FloatValue {
pub raw: Option<f64>,
}
#[derive(Debug, Deserialize)]
pub struct IntValue {
pub raw: Option<i64>,
}
#[derive(Debug, Deserialize)]
pub struct UIntValue {
pub raw: Option<u64>,
}

View file

@ -9,7 +9,7 @@ use serde::{Deserialize, Serialize};
use crate::error::IdxError; use crate::error::IdxError;
const SCHEMA_VERSION: u32 = 1; const CURRENT_SCHEMA_VERSION: u32 = 1;
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
pub struct Cache { pub struct Cache {
@ -83,7 +83,7 @@ impl Cache {
let entry = CacheEntry { let entry = CacheEntry {
fetched_at: Utc::now(), fetched_at: Utc::now(),
ttl_secs, ttl_secs,
schema_version: SCHEMA_VERSION, schema_version: CURRENT_SCHEMA_VERSION,
data, data,
}; };
let raw = serde_json::to_string_pretty(&entry) let raw = serde_json::to_string_pretty(&entry)
@ -124,17 +124,21 @@ impl Cache {
}) })
} }
pub fn clear(&self) -> Result<usize, IdxError> { pub fn clear(&self) -> Result<(usize, Vec<PathBuf>), IdxError> {
if !self.root.exists() { if !self.root.exists() {
return Ok(0); return Ok((0, Vec::new()));
} }
let mut removed = 0usize; let mut removed = 0usize;
let mut failed = Vec::new();
self.walk(&self.root, &mut |p| { self.walk(&self.root, &mut |p| {
if p.is_file() && fs::remove_file(p).is_ok() { if p.is_file() {
removed += 1; match fs::remove_file(p) {
Ok(_) => removed += 1,
Err(_) => failed.push(p.to_path_buf()),
}
} }
})?; })?;
Ok(removed) Ok((removed, failed))
} }
fn walk<F: FnMut(&Path)>(&self, dir: &Path, f: &mut F) -> Result<(), IdxError> { fn walk<F: FnMut(&Path)>(&self, dir: &Path, f: &mut F) -> Result<(), IdxError> {
@ -159,8 +163,19 @@ impl Cache {
if !path.exists() { if !path.exists() {
return Ok(None); return Ok(None);
} }
let raw = fs::read_to_string(path).map_err(|e| IdxError::Io(e.to_string()))?; let raw = fs::read_to_string(&path).map_err(|e| IdxError::Io(e.to_string()))?;
let entry = serde_json::from_str(&raw).map_err(|e| IdxError::ParseError(e.to_string()))?; let entry: CacheEntry<T> =
serde_json::from_str(&raw).map_err(|e| IdxError::ParseError(e.to_string()))?;
if entry.schema_version != CURRENT_SCHEMA_VERSION {
eprintln!(
"debug: cache schema mismatch for {} (got {}, expected {})",
path.display(),
entry.schema_version,
CURRENT_SCHEMA_VERSION
);
let _ = fs::remove_file(&path);
return Ok(None);
}
Ok(Some(entry)) Ok(Some(entry))
} }

View file

@ -40,8 +40,11 @@ pub fn handle(cmd: &CacheCmd) -> Result<(), IdxError> {
); );
} }
CacheSubcommand::Clear => { CacheSubcommand::Clear => {
let removed = cache.clear()?; let (removed, failed) = cache.clear()?;
println!("cleared {removed} files"); println!("cleared {removed} files");
if !failed.is_empty() {
eprintln!("warning: failed to remove {} file(s)", failed.len());
}
} }
} }
Ok(()) Ok(())

View file

@ -7,14 +7,23 @@ use crate::analysis::fundamental::{
}; };
use crate::analysis::signals::{self, Signal, TechnicalSignal}; use crate::analysis::signals::{self, Signal, TechnicalSignal};
use crate::analysis::technical; use crate::analysis::technical;
use crate::api::MarketDataProvider; use crate::api::msn::MsnProvider;
use crate::api::types::{Fundamentals, Interval, Ohlc, Period}; use crate::api::types::{
CompanyProfile, EarningsReport, FinancialStatements, Fundamentals, InsightData, Interval,
NewsItem, Ohlc, Period, Quote, SentimentData,
};
use crate::api::{
EarningsProvider, FinancialsProvider, InsightsProvider, MarketDataProvider, NewsProvider,
ProfileProvider, SentimentProvider, history_provider,
};
use crate::cache::Cache; use crate::cache::Cache;
use crate::config::IdxConfig; use crate::config::IdxConfig;
use crate::error::IdxError; use crate::error::IdxError;
use crate::output::{ use crate::output::{
MacdSnapshot, TechnicalReport, VolumeSnapshot, render_compare, render_fundamental, MacdSnapshot, TechnicalReport, VolumeSnapshot, render_compare, render_earnings,
render_growth, render_history, render_quotes, render_risk, render_technical, render_valuation, render_financials, render_fundamental, render_growth, render_history, render_insights,
render_news, render_profile, render_quotes, render_risk, render_screener, render_sentiment,
render_technical, render_valuation,
}; };
struct FundamentalCacheSpec { struct FundamentalCacheSpec {
@ -95,6 +104,45 @@ pub enum StocksSubcommand {
/// Single ticker symbol (e.g. BBCA). /// Single ticker symbol (e.g. BBCA).
symbol: String, symbol: String,
}, },
#[command(about = "Get company profile")]
Profile { symbol: String },
#[command(about = "Get financial statements")]
Financials {
symbol: String,
#[arg(long, default_value = "income")]
statement: String,
},
#[command(about = "Get earnings report")]
Earnings {
symbol: String,
#[arg(long)]
annual: bool,
#[arg(long)]
quarterly: bool,
#[arg(long)]
forecast: bool,
#[arg(long)]
history: bool,
},
#[command(about = "Get crowd sentiment")]
Sentiment { symbol: String },
#[command(about = "Get AI insights")]
Insights { symbol: String },
#[command(about = "Get stock news")]
News {
symbol: String,
#[arg(long, default_value_t = 10)]
limit: usize,
},
#[command(about = "MSN screener")]
Screen {
#[arg(long, default_value = "top-performers")]
filter: String,
#[arg(long, default_value = "id")]
region: String,
#[arg(long, default_value_t = 50)]
limit: usize,
},
#[command( #[command(
about = "Compare fundamentals across stocks", about = "Compare fundamentals across stocks",
after_help = "Examples:\n idx stocks compare BBCA BBRI BMRI\n idx stocks compare BBCA,BBRI,BMRI\n idx -o json stocks compare BBCA,BBRI" after_help = "Examples:\n idx stocks compare BBCA BBRI BMRI\n idx stocks compare BBCA,BBRI,BMRI\n idx -o json stocks compare BBCA,BBRI"
@ -160,6 +208,13 @@ pub fn handle(
period, period,
interval, interval,
} => { } => {
let hist_provider = history_provider(config.provider, false).ok_or_else(|| {
IdxError::Unsupported(
"MSN does not provide price history for IDX stocks. \
Use --provider yahoo for historical data."
.into(),
)
})?;
let history_bucket = cache_bucket(config, "history"); let history_bucket = cache_bucket(config, "history");
let resolved = crate::api::resolve_symbol(symbol, &config.exchange); let resolved = crate::api::resolve_symbol(symbol, &config.exchange);
let key = format!("{}-{}", period.as_str(), interval.as_str()); let key = format!("{}-{}", period.as_str(), interval.as_str());
@ -183,7 +238,7 @@ pub fn handle(
return render_history(&resolved, &stale, &config.output); return render_history(&resolved, &stale, &config.output);
} }
match provider.history(&resolved, period, interval) { match hist_provider.history(&resolved, period, interval) {
Ok(history) => { Ok(history) => {
if !no_cache { if !no_cache {
cache.put( cache.put(
@ -210,6 +265,13 @@ pub fn handle(
} }
} }
StocksSubcommand::Technical { symbol } => { StocksSubcommand::Technical { symbol } => {
let hist_provider = history_provider(config.provider, false).ok_or_else(|| {
IdxError::Unsupported(
"MSN does not provide price history for IDX stocks. \
Use --provider yahoo for technical analysis."
.into(),
)
})?;
let technical_bucket = cache_bucket(config, "technical"); let technical_bucket = cache_bucket(config, "technical");
let resolved = crate::api::resolve_symbol(symbol, &config.exchange); let resolved = crate::api::resolve_symbol(symbol, &config.exchange);
if !no_cache if !no_cache
@ -224,7 +286,7 @@ pub fn handle(
return render_technical(&stale, &config.output, config.no_color); return render_technical(&stale, &config.output, config.no_color);
} }
match provider.history(&resolved, &Period::OneYear, &Interval::Day) { match hist_provider.history(&resolved, &Period::OneYear, &Interval::Day) {
Ok(history) => { Ok(history) => {
let report = build_technical_report(&resolved, &history)?; let report = build_technical_report(&resolved, &history)?;
if !no_cache { if !no_cache {
@ -308,6 +370,71 @@ pub fn handle(
)?; )?;
render_fundamental(&report, &config.output, config.no_color) render_fundamental(&report, &config.output, config.no_color)
} }
StocksSubcommand::Profile { symbol } => {
let resolved = crate::api::resolve_symbol(symbol, &config.exchange);
let profile: CompanyProfile = fetch_msn_only(&resolved, config.provider, || {
MsnProvider::new(false).profile(&resolved)
})?;
render_profile(&profile, &config.output)
}
StocksSubcommand::Financials {
symbol,
statement: _,
} => {
let resolved = crate::api::resolve_symbol(symbol, &config.exchange);
let financials: FinancialStatements =
fetch_msn_only(&resolved, config.provider, || {
MsnProvider::new(false).financials(&resolved)
})?;
render_financials(&financials, &config.output)
}
StocksSubcommand::Earnings {
symbol,
annual: _,
quarterly: _,
forecast: _,
history: _,
} => {
let resolved = crate::api::resolve_symbol(symbol, &config.exchange);
let earnings: EarningsReport = fetch_msn_only(&resolved, config.provider, || {
MsnProvider::new(false).earnings(&resolved)
})?;
render_earnings(&earnings, &config.output)
}
StocksSubcommand::Sentiment { symbol } => {
let resolved = crate::api::resolve_symbol(symbol, &config.exchange);
let sentiment: SentimentData = fetch_msn_only(&resolved, config.provider, || {
MsnProvider::new(false).sentiment(&resolved)
})?;
render_sentiment(&sentiment, &config.output)
}
StocksSubcommand::Insights { symbol } => {
let resolved = crate::api::resolve_symbol(symbol, &config.exchange);
let insights: InsightData = fetch_msn_only(&resolved, config.provider, || {
MsnProvider::new(false).insights(&resolved)
})?;
render_insights(&insights, &config.output)
}
StocksSubcommand::News { symbol, limit } => {
let resolved = crate::api::resolve_symbol(symbol, &config.exchange);
let news: Vec<NewsItem> = fetch_msn_only(&resolved, config.provider, || {
MsnProvider::new(false).news(&resolved, *limit)
})?;
render_news(&news, &config.output)
}
StocksSubcommand::Screen {
filter,
region,
limit,
} => {
let msn = MsnProvider::new(false);
let filter_key = screener_filter_key(filter);
let region_key = screener_region_key(region);
let quotes: Vec<Quote> = fetch_msn_only("screen", config.provider, || {
msn.screener(filter_key, region_key, *limit)
})?;
render_screener(&quotes, &config.output, config.no_color)
}
StocksSubcommand::Compare { symbols } => { StocksSubcommand::Compare { symbols } => {
let mut reports: Vec<FundamentalReport> = Vec::new(); let mut reports: Vec<FundamentalReport> = Vec::new();
let mut last_error = None; let mut last_error = None;
@ -345,6 +472,40 @@ pub fn handle(
} }
} }
#[allow(dead_code)] // wired up once per-subcommand handlers are fully split
pub(crate) fn fetch_with_cache<T, F>(
cache: &Cache,
bucket: &str,
key: &str,
ttl_secs: u64,
offline: bool,
no_cache: bool,
fetch_fn: F,
) -> Result<T, IdxError>
where
T: Serialize + DeserializeOwned,
F: FnOnce() -> Result<T, IdxError>,
{
if !no_cache
&& !offline
&& let Some(cached) = cache.get::<T>(bucket, key)?
{
return Ok(cached);
}
if offline {
return cache
.get_stale::<T>(bucket, key)?
.ok_or_else(|| IdxError::Offline("no cached data available".to_string()));
}
let data = fetch_fn()?;
if !no_cache {
let _ = cache.put(bucket, key, &data, ttl_secs);
}
Ok(data)
}
fn fetch_fundamental_analysis_report<T, F>( fn fetch_fundamental_analysis_report<T, F>(
cache: &Cache, cache: &Cache,
provider: &dyn MarketDataProvider, provider: &dyn MarketDataProvider,
@ -465,6 +626,44 @@ fn average_last(values: &[f64], period: usize) -> Option<f64> {
Some(values[start..].iter().sum::<f64>() / period as f64) Some(values[start..].iter().sum::<f64>() / period as f64)
} }
fn fetch_msn_only<T>(
symbol: &str,
provider: crate::config::ProviderKind,
f: impl FnOnce() -> Result<T, IdxError>,
) -> Result<T, IdxError> {
if !matches!(provider, crate::config::ProviderKind::Msn) {
return Err(IdxError::Unsupported(format!(
"{symbol}: command requires --provider msn"
)));
}
f()
}
fn screener_filter_key(filter: &str) -> &'static str {
match filter {
"top-performers" => "st_list_topperfs",
"worst-performers" => "st_list_poorperfs",
"high-dividend" => "st_list_highdividend",
"low-pe" => "st_list_lowpe",
"52w-high" => "st_list_52wkhi",
"52w-low" => "st_list_52wklow",
"high-volume" => "st_list_highvol",
"large-cap" => "st_list_largecap",
_ => "st_list_topperfs",
}
}
fn screener_region_key(region: &str) -> &'static str {
match region {
"id" => "st_reg_id",
"us" => "st_reg_us",
"sg" => "st_reg_sg",
"hk" => "st_reg_hk",
"jp" => "st_reg_jp",
_ => "st_reg_id",
}
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use chrono::{Days, NaiveDate}; use chrono::{Days, NaiveDate};

View file

@ -92,8 +92,13 @@ impl IdxConfig {
if let Ok(output) = std::env::var("IDX_OUTPUT") { if let Ok(output) = std::env::var("IDX_OUTPUT") {
cfg.output = if output.eq_ignore_ascii_case("json") { cfg.output = if output.eq_ignore_ascii_case("json") {
OutputFormat::Json OutputFormat::Json
} else { } else if output.eq_ignore_ascii_case("table") {
OutputFormat::Table OutputFormat::Table
} else {
return Err(IdxError::ConfigError(format!(
"invalid IDX_OUTPUT value: '{}', expected 'json' or 'table'",
output
)));
}; };
} }
if let Ok(no_color) = std::env::var("IDX_NO_COLOR") { if let Ok(no_color) = std::env::var("IDX_NO_COLOR") {

View file

@ -16,12 +16,16 @@ pub enum IdxError {
ParseError(String), ParseError(String),
#[error("cache miss: {0}")] #[error("cache miss: {0}")]
CacheMiss(String), CacheMiss(String),
#[error("offline: {0}")]
Offline(String),
#[error("config error: {0}")] #[error("config error: {0}")]
ConfigError(String), ConfigError(String),
#[error("io error: {0}")] #[error("io error: {0}")]
Io(String), Io(String),
#[error("http error: {0}")] #[error("http error: {0}")]
Http(String), Http(String),
#[error("auth error: {0}")]
AuthError(String),
} }
#[derive(Debug, Clone, Copy, Serialize, PartialEq, Eq)] #[derive(Debug, Clone, Copy, Serialize, PartialEq, Eq)]
@ -32,9 +36,11 @@ pub enum ErrorCode {
Unsupported, Unsupported,
ParseError, ParseError,
CacheMiss, CacheMiss,
Offline,
ConfigError, ConfigError,
Io, Io,
Http, Http,
AuthError,
} }
impl IdxError { impl IdxError {
@ -46,9 +52,11 @@ impl IdxError {
Self::Unsupported(_) => ErrorCode::Unsupported, Self::Unsupported(_) => ErrorCode::Unsupported,
Self::ParseError(_) => ErrorCode::ParseError, Self::ParseError(_) => ErrorCode::ParseError,
Self::CacheMiss(_) => ErrorCode::CacheMiss, Self::CacheMiss(_) => ErrorCode::CacheMiss,
Self::Offline(_) => ErrorCode::Offline,
Self::ConfigError(_) => ErrorCode::ConfigError, Self::ConfigError(_) => ErrorCode::ConfigError,
Self::Io(_) => ErrorCode::Io, Self::Io(_) => ErrorCode::Io,
Self::Http(_) => ErrorCode::Http, Self::Http(_) => ErrorCode::Http,
Self::AuthError(_) => ErrorCode::AuthError,
} }
} }

View file

@ -7,7 +7,10 @@ use serde::{Deserialize, Serialize};
use crate::analysis::fundamental::{FundamentalReport, GrowthReport, RiskReport, ValuationReport}; use crate::analysis::fundamental::{FundamentalReport, GrowthReport, RiskReport, ValuationReport};
use crate::analysis::signals::TechnicalSignal; use crate::analysis::signals::TechnicalSignal;
use crate::api::types::{Ohlc, Quote}; use crate::api::types::{
CompanyProfile, EarningsReport, FinancialStatements, InsightData, NewsItem, Ohlc, Quote,
SentimentData,
};
use crate::error::IdxError; use crate::error::IdxError;
#[derive(Debug, Clone, Copy, ValueEnum, PartialEq, Eq, Serialize, serde::Deserialize, Default)] #[derive(Debug, Clone, Copy, ValueEnum, PartialEq, Eq, Serialize, serde::Deserialize, Default)]
@ -137,6 +140,62 @@ pub fn render_compare(
} }
} }
pub fn render_profile(profile: &CompanyProfile, format: &OutputFormat) -> Result<(), IdxError> {
match format {
OutputFormat::Table => table::print_profile(profile),
OutputFormat::Json => json::print_json(profile),
}
}
pub fn render_financials(
financials: &FinancialStatements,
format: &OutputFormat,
) -> Result<(), IdxError> {
match format {
OutputFormat::Table => table::print_financials(financials),
OutputFormat::Json => json::print_json(financials),
}
}
pub fn render_earnings(report: &EarningsReport, format: &OutputFormat) -> Result<(), IdxError> {
match format {
OutputFormat::Table => table::print_earnings(report),
OutputFormat::Json => json::print_json(report),
}
}
pub fn render_sentiment(data: &SentimentData, format: &OutputFormat) -> Result<(), IdxError> {
match format {
OutputFormat::Table => table::print_sentiment(data),
OutputFormat::Json => json::print_json(data),
}
}
pub fn render_insights(data: &InsightData, format: &OutputFormat) -> Result<(), IdxError> {
match format {
OutputFormat::Table => table::print_insights(data),
OutputFormat::Json => json::print_json(data),
}
}
pub fn render_news(items: &[NewsItem], format: &OutputFormat) -> Result<(), IdxError> {
match format {
OutputFormat::Table => table::print_news(items),
OutputFormat::Json => json::print_json(items),
}
}
pub fn render_screener(
quotes: &[Quote],
format: &OutputFormat,
no_color: bool,
) -> Result<(), IdxError> {
match format {
OutputFormat::Table => table::print_quotes(quotes, no_color),
OutputFormat::Json => json::print_json(quotes),
}
}
pub fn emit_error(err: &IdxError, format: &OutputFormat) { pub fn emit_error(err: &IdxError, format: &OutputFormat) {
match format { match format {
OutputFormat::Table => eprintln!("Error: {err}"), OutputFormat::Table => eprintln!("Error: {err}"),

View file

@ -3,7 +3,10 @@ use owo_colors::OwoColorize;
use crate::analysis::fundamental::{FundamentalReport, GrowthReport, RiskReport, ValuationReport}; use crate::analysis::fundamental::{FundamentalReport, GrowthReport, RiskReport, ValuationReport};
use crate::analysis::signals::Signal; use crate::analysis::signals::Signal;
use crate::api::types::{Ohlc, Quote}; use crate::api::types::{
CompanyProfile, EarningsData, EarningsReport, FinancialStatements, InsightData, NewsItem, Ohlc,
Quote, SentimentData,
};
use crate::error::IdxError; use crate::error::IdxError;
use crate::output::TechnicalReport; use crate::output::TechnicalReport;
@ -516,6 +519,194 @@ fn add_compare_row(table: &mut Table, label: &str, values: Vec<String>) {
table.add_row(row); table.add_row(row);
} }
pub fn print_profile(profile: &CompanyProfile) -> Result<(), IdxError> {
let mut table = Table::new();
table
.load_preset(UTF8_FULL)
.set_header(vec!["FIELD", "VALUE"]);
// Use long_name with short_name as fallback (IDX stocks often only have shortName)
let name = if !profile.long_name.is_empty() {
&profile.long_name
} else {
&profile.short_name
};
let add_if_present = |t: &mut Table, label: &str, value: &str| {
if !value.is_empty() {
t.add_row(vec![Cell::new(label), Cell::new(value)]);
}
};
add_if_present(&mut table, "Symbol", &profile.symbol);
add_if_present(&mut table, "Name", name);
add_if_present(&mut table, "Sector", &profile.sector);
add_if_present(&mut table, "Industry", &profile.industry);
add_if_present(&mut table, "Website", &profile.website);
add_if_present(&mut table, "Country", &profile.country);
add_if_present(&mut table, "City", &profile.city);
add_if_present(&mut table, "Phone", &profile.phone);
if profile.employees > 0 {
table.add_row(vec![
Cell::new("Employees"),
Cell::new(profile.employees.to_string()),
]);
}
if !profile.description.is_empty() {
// Truncate long descriptions for table display
let desc = if profile.description.len() > 200 {
format!("{}...", &profile.description[..200])
} else {
profile.description.clone()
};
table.add_row(vec![Cell::new("Description"), Cell::new(desc)]);
}
if !profile.officers.is_empty() {
table.add_row(vec![
Cell::new("Executives"),
Cell::new(
profile
.officers
.iter()
.take(5)
.map(|o| format!("{} ({})", o.name, o.title))
.collect::<Vec<_>>()
.join("\n"),
),
]);
}
println!("{table}");
Ok(())
}
pub fn print_financials(fin: &FinancialStatements) -> Result<(), IdxError> {
let print_section = |label: &str, section: &crate::api::types::StatementSection| {
println!("\n── {label} ({}) ──", section.end_date);
let mut t = Table::new();
let value_header = format!("VALUE ({})", section.currency);
t.load_preset(UTF8_FULL)
.set_header(vec!["LINE ITEM", value_header.as_str()]);
// Sort keys for deterministic output
let mut entries: Vec<(&String, &f64)> = section.values.iter().collect();
entries.sort_by_key(|(k, _)| k.as_str());
for (k, v) in entries {
t.add_row(vec![Cell::new(k), Cell::new(format_idr(*v as i64))]);
}
println!("{t}");
};
if let Some(income) = &fin.income_statement {
print_section("Income Statement", income);
}
if let Some(balance) = &fin.balance_sheet {
print_section("Balance Sheet", balance);
}
if let Some(cf) = &fin.cash_flow {
print_section("Cash Flow", cf);
}
if fin.income_statement.is_none() && fin.balance_sheet.is_none() && fin.cash_flow.is_none() {
println!("No financial statement data available for this stock.");
}
Ok(())
}
pub fn print_earnings(report: &EarningsReport) -> Result<(), IdxError> {
let mut table = Table::new();
table.load_preset(UTF8_FULL).set_header(vec![
"PERIOD",
"EPS ACT",
"EPS FC",
"SURPRISE",
"SURPRISE%",
"REVENUE",
"DATE",
]);
for row in &report.history {
add_earnings_row(&mut table, row);
}
for row in &report.forecast {
add_earnings_row(&mut table, row);
}
println!("{table}");
Ok(())
}
pub fn print_sentiment(data: &SentimentData) -> Result<(), IdxError> {
let mut table = Table::new();
table
.load_preset(UTF8_FULL)
.set_header(vec!["RANGE", "BULLISH", "BEARISH", "NEUTRAL"]);
for row in &data.statistics {
table.add_row(vec![
Cell::new(&row.time_range),
Cell::new(row.bullish),
Cell::new(row.bearish),
Cell::new(row.neutral),
]);
}
println!("{table}");
Ok(())
}
pub fn print_insights(data: &InsightData) -> Result<(), IdxError> {
println!("{}", data.summary);
if !data.highlights.is_empty() {
println!("Highlights:");
for h in &data.highlights {
println!("- {h}");
}
}
if !data.risks.is_empty() {
println!("Risks:");
for r in &data.risks {
println!("- {r}");
}
}
Ok(())
}
pub fn print_news(items: &[NewsItem]) -> Result<(), IdxError> {
let mut table = Table::new();
table
.load_preset(UTF8_FULL)
.set_header(vec!["TITLE", "PROVIDER", "DATE", "URL"]);
for item in items {
table.add_row(vec![
Cell::new(&item.title),
Cell::new(&item.provider),
Cell::new(&item.published_at),
Cell::new(truncate_url(&item.url)),
]);
}
println!("{table}");
Ok(())
}
fn add_earnings_row(table: &mut Table, row: &EarningsData) {
table.add_row(vec![
Cell::new(&row.period_type),
Cell::new(format_float(row.eps_actual, 2)),
Cell::new(format_float(row.eps_forecast, 2)),
Cell::new(format_float(row.eps_surprise, 2)),
Cell::new(format_float(row.eps_surprise_pct, 2)),
Cell::new(format_float(row.revenue_actual, 2)),
Cell::new(
row.earning_release_date
.clone()
.unwrap_or_else(|| "-".to_string()),
),
]);
}
fn truncate_url(url: &str) -> String {
if url.len() > 72 {
format!("{}...", &url[..72])
} else {
url.to_string()
}
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::{format_idr, format_signal, format_u64}; use super::{format_idr, format_signal, format_u64};

View file

@ -107,27 +107,31 @@ fn technical_with_mock_provider_json_contains_fields() {
} }
#[test] #[test]
fn msn_history_reports_unsupported() { fn msn_history_returns_unsupported() {
// MSN Finance/Charts returns 404 for IDX (XIDX) stocks — history is not supported.
// history_provider() returns None for MSN, which surfaces as Unsupported error.
test_bin("msn-history-unsupported") test_bin("msn-history-unsupported")
.env("IDX_PROVIDER", "msn") .env("IDX_PROVIDER", "msn")
.args(["stocks", "history", "BBCA", "--period", "1mo"]) .env("IDX_USE_MOCK_PROVIDER", "1")
.args(["stocks", "history", "BBCA", "--period", "3mo"])
.assert() .assert()
.failure() .failure()
.stderr(predicate::str::contains( .stderr(predicate::str::contains(
"MSN provider does not currently support history or technical analysis", "MSN does not provide price history",
)); ));
} }
#[test] #[test]
fn msn_technical_json_reports_unsupported() { fn msn_technical_returns_unsupported() {
// Technical analysis requires history — also unsupported for MSN/IDX.
test_bin("msn-technical-unsupported") test_bin("msn-technical-unsupported")
.env("IDX_PROVIDER", "msn") .env("IDX_PROVIDER", "msn")
.env("IDX_USE_MOCK_PROVIDER", "1")
.args(["-o", "json", "stocks", "technical", "BBCA"]) .args(["-o", "json", "stocks", "technical", "BBCA"])
.assert() .assert()
.failure() .failure()
.stderr(predicate::str::contains("\"code\": \"UNSUPPORTED\""))
.stderr(predicate::str::contains( .stderr(predicate::str::contains(
"MSN provider does not currently support history or technical analysis", "MSN does not provide price history",
)); ));
} }