Commit graph

40 commits

Author SHA1 Message Date
c03c0f41f7 fix(quote): market_status reads open or close with a login too
Logged-in quotes passed Stockbit's session state through verbatim
(STATE_NAME_SESSION_2), while public-page quotes say open or close, so a
consumer checking for "open" broke after login. Quotes now use the two
words either way; the order book keeps the state verbatim.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 17:31:20 +07:00
63587e4dc2 fix(history): fall back to Yahoo when Stockbit refuses the session
auto history picks Stockbit when a session file exists, so an expired
session failed history and technical with AUTHERROR. auto now retries on
Yahoo with a warning; an explicit --history-provider stockbit still fails.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 17:31:20 +07:00
e7f1fa2c90 fix(stockbit): keep a working session through a failed login check
- login restored the previous session on any check error, even after the
  check had rotated the pasted session: a timeout then left no working
  session. It now keeps a rotated session unless Stockbit refused it.
- The browser sign-in marked a cookie refused on any error, so a 429 or a
  timeout held a signed-in user until the wait ran out. Passing errors
  are retried after a pause; only a refusal is final.
- The local page checked concurrent pastes on their own threads, so a
  later one could overwrite the session the first had just rotated.
  Pastes are now checked one at a time, and none after one is accepted.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 17:31:20 +07:00
32f476bae7 fix(stockbit): bid/offer while trading; drop stale foreign flow
Observed in session on 2026-11-03:

- The regular board trades as STATE_NAME_SESSION_2, not the assumed
  STATE_NAME_MARKET_OPEN, so session quotes hid bid/offer all day. Any
  numbered STATE_NAME_SESSION_<n> now counts as trading; the lunch break
  is STATE_NAME_BREAK.
- In session the order book's fbuy/fsell/fnet were the previous session's
  (equal to IPOT's 2026-11-02 net for BBCA, BBRI and TLKM) and
  has_foreign_bs was false. Foreign figures are kept only when that flag
  is true.

Fixtures for the break, the trading state and an in-session book; the
table names the session ("market open (session 2)").

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-03 14:02:03 +07:00
ca4d3d09c2 feat(stockbit): one browser sign-in in idx-core for the CLI and MCP
`stockbit::signin` is now the single sign-in flow. `idx stockbit login`
waits on it in the foreground; the `stockbit_login` MCP tool runs it on a
thread and `stockbit_status` reports `sign_in.state`, so an agent knows when
to stop polling. Auto uses a browser unless the shell is over SSH or has no
display, and falls back to the local page only when the browser cannot start.

The browser way opens Stockbit's sign-in page in a profile only idx uses and
reads the session cookie over DevTools. What a real sign-in showed:

- The cookie is host-only on stockbit.com, so deleting by the assumed
  .stockbit.com removed nothing and left live tokens in the profile. It is
  now deleted by its own domain and path, with every other tab closed first,
  and read back to confirm.
- A signed-out page writes the cookie without tokens; it no longer counts.
- A profile belongs to one browser, so it records the executable that made it.
- On macOS the config directory depends on XDG_CONFIG_HOME, so a shell and an
  app started from the Dock disagreed; both default locations are now checked.
- A thread does not outlive main, so idx-mcp closes a waiting browser when
  stdin closes or on SIGTERM or Ctrl-C.

Progress goes to stderr, which keeps `-o json` to one document on stdout.

Google refused its own sign-in in this window on 2026-10-31, most likely over
the debugging flag; password sign-in works, and the docs and roadmap say so.

Claude-Session: https://claude.ai/code/session_01RqPPVaoFxLToGAXXBEoCtH
2026-10-31 23:58:21 +07:00
hermes
cb532b670b feat(stockbit): print the ssh forward when the shell is remote
The login page listens on 127.0.0.1, so a browser on the Mac driving a VM
could never reach it. Over SSH, idx now picks port 7878, skips opening a
browser on the VM, and prints the exact ssh -L to run where the browser
is.
2026-10-23 13:52:15 +00:00
hermes
00c59ad9c9 fix(stockbit): answer login-page connections concurrently
Browsers open idle preconnects; serving one connection at a time made the
page wait out the 3 s read timeout on each, which read as "can't
connect". Each connection now gets its own thread (capped) and a silent
one is dropped. Regression test; measured 10.1 s -> 0.08 s with three
idle sockets. Docs cover a browser on another machine (--port + ssh -L).
2026-10-23 13:28:12 +00:00
hermes
12fdc8c8e8 feat(stockbit): connect a session from a local browser page
idx stockbit login in a terminal now serves a page on 127.0.0.1: a
bookmarklet that reads stockbit.com's session cookie in the user's own
browser and opens the page with it in the URL fragment, or a
read-the-clipboard button, with the manual paste as the fallback. A
per-run secret guards the path, the page is no-store with a strict CSP,
and it stops on success or after five minutes. --no-browser prints the
URL (--port pins it for ssh -L); piped stdin keeps the paste path.

A paste Stockbit refuses now restores the previous session instead of
losing it.
2026-10-23 12:29:01 +00:00
hermes
b5ae9b98de feat: Stockbit login for the order book, live quotes and daily history
Opt-in, with the user's own Stockbit session from a pasted
credentialStorage cookie (or IDX_STOCKBIT_REFRESH_TOKEN):

- idx stockbit login/status/logout; session file 0600, tokens never shown
- rotating refresh under a cross-process lock; one refresh+retry on 401
- stocks orderbook: depth with lots and orders, OHLC, value, foreign flow
- --provider stockbit quotes from the session API when logged in
- --history-provider stockbit, and auto under the stockbit provider for
  5d-1y daily windows
- MCP get_orderbook and stockbit_status (17 tools)

Endpoints, units and recorded responses from fzn0x/idx (MIT).
2026-10-23 08:35:57 +00:00
hermes
74e5bb0536 feat: login-free Stockbit intraday and quote provider
Read Stockbit's public symbol page (__NEXT_DATA__) with a plain ureq
request and an honest user agent: `stocks intraday` (today by the
minute, any provider), `--provider stockbit` quotes with the best
bid/offer only while the market is open, and MCP `get_intraday`.

- UPSTREAMBLOCKED error (exit 4) for a CDN 403; never retried
- requests paced 1 s apart across idx processes, 5 s snapshot cache
- global --provider flag (yahoo, msn, stockbit)
- roadmap: L1 ships from Stockbit; depth stays a login follow-on
2026-10-23 04:58:32 +00:00
hermes
834e47b336 fix: mark only IDX quotes behind the session; test the wiring
Review findings on #71:
- The IDX session rule flagged non-IDX data: US screen rows on Monday
  evening WIB, and JP/HK/SG rows on their own holidays. Only `.JK`
  quotes and `--region id` screens are marked now.
- Nothing tested the marking call sites. A screen test checks IDX rows
  online and offline, and US rows unmarked. A quote test reads a cache
  entry in 0.3.3's format (no field) offline.
- The get_quotes description now says the flag also means the stock did
  not trade, or an exchange holiday; screen_stocks mentions the field.
- latest_session is pub(crate).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-19 10:26:09 +00:00
hermes
a0a2cac7cf feat: flag quotes behind the latest IDX session (#50)
MSN can serve the previous session's close for hours after IDX closes;
on 2026-10-16 at 16:53 WIB it still returned the 15th. Quotes and
screener rows now carry `behind_session`, true when `as_of` is before
the latest weekday whose session has ended (16:15 WIB). It is computed
on return, not cached. The quote table notes such quotes, and the
get_quotes description explains the field.

Closes #50

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 17:38:48 +00:00
hermes
7d55be80c5 fix(core): bind IPOT class tokens to the elements that carry them
Classes are allowed per element type, not page-wide, and the table and
each footer element on the way to a total must carry exactly IPOT's
classes, so Bootstrap's visible-xs (hidden at desktop widths) on a total
or the table rejects the page (Pullfrog review).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 14:35:02 +00:00
hermes
3c5f9bb8a4 fix(core): an IPOT page without a readable date range is rejected
Ancestor classes compare as sets (repeats removed too), and a page whose
echoed range can't be read is a parse error rather than a summary with
no dates, so the parser returns the exact range or nothing (Codex
re-check, round 13). The broker layer already rejected missing dates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 14:25:45 +00:00
hermes
bf77073877 fix(core): close the round-12 audit of the IPOT parser
From Codex's exhaustive audit of every allowed element, attribute and
returned value:
- class tokens split on ASCII whitespace as HTML does, so a no-break
  space stays in the token (and fails the class allowlist);
- a date input's line breaks are dropped before parsing, as a text input
  drops them;
- ancestor placement compares class token sets, not exact strings;
- the echoed range needs both date inputs, start then end in document
  order, each with IPOT's id and name; otherwise no dates;
- a side with a blank broker code may show no figures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 14:15:51 +00:00
hermes
d15fc4833a fix(core): header cells take only align; broker kind from the class token
A header cell spanning columns would move the labels over the figures,
so header cells may carry only align. The broker kind comes from the
span's single class token as the browser splits it, so spacing in the
attribute no longer drops it (Codex re-check, round 11).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 13:58:59 +00:00
hermes
828d16c92e fix(core): match IPOT's unknown-code reply exactly; check column labels
An unknown code is only the reply that is exactly "Invalid stock code!",
not any page with that text (e.g. inside a script). The header row must
be IPOT's nine labels in order, so a page labelling the columns otherwise
is rejected rather than read by position (Codex re-check, round 10).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 13:49:15 +00:00
hermes
ee4fe4b852 fix(core): read IPOT's four footer spans exactly; bound the page size
The footer div must hold exactly IPOT's four text-only spans, in order and
each starting with its label, with nothing else shown between them, so a
total split from or across spans no longer reads as a smaller figure.
Pages over 64 KiB or 2,000 tags (IPOT's is about 8 KB and under 500) are
rejected before parsing, which bounds tree building on deep nesting
(Codex re-check, round 9).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 13:40:01 +00:00
hermes
63c4d9342f fix(core): accept only IPOT's own markup, in its own places
Replace the list of hiding mechanisms with what the page is: every element
must be one IPOT uses, with only its attributes and class tokens, and with
input type, the footer div's style, colspan and button type pinned; the
table and date inputs must sit at IPOT's exact div paths; the table is
thead, tbody, tfoot; totals come from tfoot > tr > th > div > span only;
and a cell's span holds all the cell shows. A popover, an escaped inline
style, a hidden input, a total in details or a code beside its span now
reject the page, and the footer is walked directly (linear). Tests now
state the property: a page is rejected or read as shown, never as other
data (Codex re-check, round 8).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 13:30:08 +00:00
hermes
723da4a5a6 fix(core): the IPOT table and date inputs must sit in plain divs
Inside details, dialog, noscript, object, select and the like they may
not show, so the table there rejects the page and such a date input is
no echoed date.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 13:18:30 +00:00
hermes
6cfceafe04 fix(core): read IPOT cells and footer only in the shape IPOT writes
On the html5ever DOM, a body cell must be a td (align only) holding text
or one span with a single text-* class holding text; footer totals must be
text-only spans, each label once; each date input must be unique; and a
template or an element hidden by markup (hidden, inline display:none or
visibility:hidden) anywhere rejects the page. So script or hidden text
inside a figure, a hidden total, a decoy span before a broker code, or a
duplicate date input can no longer be read as data, and footer text is
read once per span (Codex re-check, round 7).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 13:09:13 +00:00
hermes
79b15d51b9 fix(core): parse IPOT pages with html5ever
Six rounds of the 0.4.0 review found crafted markup that the hand-written
reader read differently from HTML (cells in comments or attributes, stray
or spaced tags, a decoy date attribute or total, foreign content). Parse
the page with scraper (html5ever) instead: dates are the inputs' values,
rows are the single tbody's tr children with nine td each, totals are the
tfoot spans, all as a browser builds them. Rows that don't line up still
reject the page. scraper's CSS crates (selectors, cssparser) are MPL-2.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 12:54:34 +00:00
hermes
1c6ec6eb28 fix(core): read the IPOT page as a browser would around its table
Comments are removed before parsing, and an unclosed or abrupt one
(<!-->, <!--->, --!>) or other <! / <? markup rejects the page; the page
is read only up to its first raw-text element (it ends with one empty
script). The echoed dates are read before the table and the totals after
it, cell text may not hold '>', and the row loop no longer rescans
trailing whitespace per row (Codex re-check, round 5).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 12:34:16 +00:00
hermes
cecfbab235 fix(core): accept only the exact markup IPOT serves in its table body
Replace the tag scanner with a strict grammar: whole <tr> rows of nine
<td> cells, each holding text or one <span>, attributes as name="value"
without quotes or angle brackets in the value, and no character references
besides &nbsp;. Markup in that form reads the same in any HTML parser, so
a quoted '>', '</tr >', '<td-x' or '&#50;' can no longer make the parser
see other cells than the page shows; such pages are rejected (Codex
re-check, round 4).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 12:24:44 +00:00
hermes
186d9557d6 fix(core): the IPOT table body holds plain row markup only
One tbody start and end in the page, in any case, and inside it only tr,
td and span tags with no '<' before a tag closes. A stray </TbOdY> no
longer cuts the table short, and cells inside a comment or an attribute
are no longer read as brokers (Codex re-check).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 12:15:40 +00:00
hermes
30afcc109d fix(core): read the IPOT table body as whole rows only
The body (one tbody, tags matched case-insensitively) must be whole rows:
each one <tr> holding nine opened and nine closed cells, ranked in order,
with nothing else between rows. A row with missing close tags no longer
merges into its neighbour, and an uppercase row is no longer skipped
(Codex re-check).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 12:05:13 +00:00
hermes
106590d151 fix(core): a row with no parsed cells rejects the page too
Every piece of the table body that holds a row must parse as nine cells;
ranks count the rows read, not the pieces. A rank-1 row without its cell
ends was skipped and ranks 2-10 still matched (Codex re-review).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 11:56:25 +00:00
hermes
d82f7c6be9 fix(core): reject broker and trading pages that would misreport a session
From the 0.4.0 release review (Codex):
- IPOT pages with a short row, a rank out of order or a broker after a
  blank row are parse errors; skipping a row shifted later brokers up a
  rank and could flip a bandar reading (-2425 bps to -66 bps).
- A tbody end before its start is a parse error, not a panic.
- An IPOT page that echoes other dates than requested is rejected before
  it is cached, so one session's figures never pass as another's.
- An IDX trading summary with rows for other days only is an error, not
  an empty session cached for 30 days.
- Empty sessions count toward a flow's fetched_at and stale.
- The bandar imbalance uses checked i128 arithmetic: past i128 there is
  no reading instead of a wrapped one.
- The bandar table prints "No trades." only when no broker traded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 11:48:00 +00:00
hermes
29f143fa1b feat: foreign flow (idx stocks flow, get_foreign_flow)
`idx stocks flow SYMBOL` shows what foreign investors bought and sold of a
stock, session by session, from IDX's own daily trading summary: foreign
buy, sell and net volume, net value at the session's average price, and
the share of value traded outside the regular board, with range totals.
Latest 10 sessions by default (--days, at most 60) or --from/--to. The MCP
server exposes it as get_foreign_flow.

- api/idx: pure parser for GetStockSummary (rows dated another session
  are dropped) and a provider over curl-impersonate's profile fallback
  with a 30 s limit; Cloudflare passes only some fingerprints.
- flow: each session is cached whole by date, so other stocks' flows for
  the same sessions need no request. Weekends are never requested.
- Idx::idx_code is shared by broker and flow for the .JK-only check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 05:39:12 +00:00
hermes
2c93c73b6a feat(core): broker summaries from IPOT
Adds a BrokerSummaryProvider capability served by Indo Premier's public
broker summary page: the top 10 buyers and sellers of a stock by lots over
a date range, with totals. No login or key.

- api/ipot: pure parser (decimal scaling of `1.6 M`-style figures), a
  client that retries the server's spurious 404s, and a fixture mock.
- broker: Broker::summary/latest over the cache (7 days for closed ranges,
  5 minutes for today), range checks, and IDX member names bundled from
  GetBrokerCodeList.
- foreign_net_value is null for filtered summaries: the source does not
  apply the filters to it.

The endpoint's behavior was first mapped by fzn0x/idx (MIT).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 04:28:38 +00:00
hermes
49b2d3aaba fix(analysis): ratings need evidence; negative equity is high risk
From the full-repo Codex audit:
- The overall fundamental rating was "healthy" whenever growth was
  growing and valuation and risk were merely not "expensive" / not "high
  risk", so with no valuation or risk data at all (partial MSN
  fundamentals) it still read "healthy". It now needs valuation
  undervalued or fairly valued and risk low or moderate.
- Risk recognized only heavy leverage or a weak current ratio as high
  risk; negative equity fell through to "moderate risk", and a known
  adverse signal with the other metric missing became "incomplete data".
  Any known adverse signal now makes it high risk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 02:16:18 +00:00
hermes
46cf69db6e fix(quotes): keep negative price changes
From the Codex re-gate of the 0.3.3 fixes: the price sanity check also
applied to MSN's priceChange, so a falling stock without a previous close
got change: null. Bound a change by its size instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-17 12:26:26 +00:00
hermes
66207d24b6 fix(net): bounded request time, curl size and time caps, lock waits, sane prices
From the full-repo Codex audit:
- MSN and Yahoo clients set connect and body timeouts but none for the
  response headers, so a server that stalled before answering hung the
  call; four such calls take every MCP call slot. Add a 30 s overall
  deadline. Every curl-impersonate run gets --max-time 600 and a 256 MiB
  --max-filesize, and output above that is rejected (curl cannot cap a
  response without a length); the Yahoo cookie fetch gets --max-time.
- Ownership commands blocked on the database lock without limit, so a
  long sync or import, or a stuck process, held MCP call slots
  indefinitely. Wait up to 30 s, then fail with a "busy, try again" error.
- A price of 1e20 saturated `as i64` and the change subtraction then
  overflowed. Prices that are not finite, negative, or above 1e12 IDR are
  treated as missing, and a quote's own price as an error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-17 12:26:26 +00:00
hermes
8c9c93339f fix(news): order news by instant, not timestamp string
String order put 10:00:00Z before the newer 10:00:00.500Z and mixed
time-zone offsets; with a small --limit the newest item could drop out.
Compare parsed RFC 3339 instants; missing or unparseable dates sort last.
From a Codex review.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-16 09:54:30 +00:00
hermes
1796e58105 feat(news): rank this company's news first and label each item
MSN's feed for a stock is often mostly general market news: of 10 items,
ADRO, BREN and CUAN had none about the company, BBRI one, yet they were
presented as the company's news. Fetch at least 50 items, label each
`relevance: company` (tagged by MSN with the stock's instrument id, or
naming the ticker or the company) or `market`, and return company items
first, newest first, trimmed to --limit.

Company names come from the quote's short and display names, best
effort: MSN tags Telkom articles with its NYSE ADR, not TLKM, so names
catch what tags miss. Classification stays a pure function in parse_news.

Output change, intentional: news JSON gains `relevance`, the table an
ABOUT column (goldens regenerated), and the get_company tool description
explains the field (tools.json and the website tools page regenerated).
News cache keys change so entries without the field are not reused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-16 09:54:30 +00:00
hermes
60634d3aef fix: address final review (path traversal, sync races, MCP limits)
Findings from a Codex (gpt-6.1-sol, xhigh) review of v0.2.4..HEAD, each
verified in the code before fixing:

- high: ticker symbols reached cache file paths unchecked, so a symbol
  like `../../REPORT` read (and deleted, if unparseable) .json files
  outside the cache, reachable through MCP get_quotes. resolve_symbol now
  accepts only ticker characters, and cache keys are encoded as single
  safe path components.
- high: sync decided before downloading and replaced the database
  without rechecking; an import that landed meanwhile was erased. It now
  rechecks the local state before installing, and idx-mcp runs
  ownership_sync exclusively with ownership reads.
- replaces_sqlite_sha256 never downgrades history: the client requires
  the rebuild to keep at least the local release count, and the
  publisher lists a replaced build only then.
- cache writes are atomic (temp file + rename), so concurrent readers
  never see, and delete, a partial entry.
- idx-mcp caps symbols (20, deduplicated) and limits, and runs at most 4
  tool calls at once.
- each Yahoo client gets its own cookie jar, paired with its crumb.
- ZIP entries (XLSX, KSEI archives) are read with a 256 MiB bound;
  non-ASCII archive dates and huge percentages are parse errors instead
  of panics or overflow.
- agent-smoke --keep deletes every copied agent config, not only auth
  files.

CLI goldens unchanged; the MCP tool golden changes only to state the
new caps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 21:58:37 +00:00
hermes
cda12fe2db feat(mcp): add idx-mcp, a stdio MCP server over idx-core
Phase 4 of docs/DESIGN.md. Eleven grouped tools (quotes, history, screen,
company, analysis, compare, shareholders, investor holdings, search,
ownership changes, status); ownership_sync is listed only with
--allow-sync. Live results keep the Fetched envelope, errors use the CLI's
{error, code, message} body, and investor tools on an empty database
point at the sync path.

idx-core gains Period/Interval FromStr, Investor::status, a ticker
filter on changes, and Display for ErrorCode (now used by the CLI too).
CLI output and goldens are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:19:04 +00:00
hermes
86cb1b4b7c refactor(core): move stocks fetching, caching, and analysis into idx-core
Phase 3b of docs/DESIGN.md. `stocks` handlers are now argument parsing,
one idx-core call, and rendering.

- Idx (client.rs) is the entry point: Idx::new(CoreConfig) with
  market(), financial(), and analysis(). Live lookups return
  Fetched<T> { data, source, fetched_at, stale } and take a CachePolicy
  (Default | Offline | Refresh, from the --offline/--no-cache flags).
- One cached_fetch replaces the CLI's three cache flows; they differed
  only in whether a fresh entry is read via get or get_offline when
  offline, which yields the same data and diagnostics. Report-building
  errors (technical) still skip the stale-cache fallback.
- market.rs: quotes, history (history_source is resolved first so the
  CLI's fallback note keeps its position), and a typed screener
  (ScreenFilter/ScreenRegion with the same error messages).
- financial.rs: profile, statements, earnings, sentiment, insights,
  news, with StatementFilter/EarningsFilter.
- analysis: TechnicalReport moves to analysis::report; Analysis offers
  technical, growth, valuation, risk, fundamental, and compare.
- CoreConfig::load() owns provider/exchange/TTL settings from
  config.toml and IDX_*; the CLI's IdxConfig adds output and color.
  With two different invalid settings at once, the reported one may
  differ from before; every single-error message is unchanged.
- Cache gains fresh/any/offline accessors returning fetch metadata.

Unit tests moved with their code (core 157 -> 168, CLI 17 -> 7). All
143 goldens unchanged; live A/B against main: 16/16 stocks identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 09:41:31 +00:00
hermes
e78d9fcecd refactor(core): compile the mock provider only with a testing feature
Phase 3a of docs/DESIGN.md. MockProvider moves to api/mock.rs, and it,
the embedded MSN fixture bodies, and the IDX_MOCK_* hooks compile only
with idx-core's new `testing` feature. idx-cli enables it as a
dev-dependency, so `cargo test` keeps the fixture-backed provider while
release binaries no longer embed fixture data or honor
IDX_USE_MOCK_PROVIDER. Parser helpers used only by the mock are gated
the same way.

The packaged-cli flake check no longer relies on the mock; it checks
an offline cache miss (JSON CACHEMISS, exit 3) instead.

Goldens unchanged; clippy is clean with and without `testing`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 09:31:12 +00:00
hermes
15947de51a refactor: split into idx-core library and idx-cli binary workspace
Phase 2 of docs/DESIGN.md, no behavior change:

- crates/idx-core (library): api, analysis, cache, config file handling
  and provider selection, curl_impersonate, error, ownership, runtime,
  plus the shared test fixtures. clap derives are behind an optional
  `clap` feature so the library does not require clap.
- crates/idx-cli (binary `idx`): cli/, output/, and IdxConfig (output
  format and color resolution). Package and binary names are unchanged.
- config.rs is split: file read/write and validation move to core; a
  CLI test keeps core's OUTPUT_VALUES in sync with OutputSetting.
- ownership::db::open_db/db_path drop their unused IdxConfig argument.
- One workspace version; shared dependency versions; no third-party
  version changes in Cargo.lock. LICENSE is symlinked into each crate.
- flake: workspace version, new source paths, and a per-crate package
  check (idx-cli is built from its tarball against packaged idx-core,
  since idx-core is not on crates.io yet). CI cache key, release path
  filter, and the golden whitespace-hook exclusion follow the new layout.

Parity: all 143 golden outputs match byte for byte without
regeneration; 157 core + 17 CLI unit tests (173 before plus the new
OUTPUT_VALUES sync test), 62 CLI tests pass. Live A/B against main:
16/16 stocks cases identical; ownership discover got IDX 403 on both
(exit 4 vs 1 is the phase 1 exit-code change).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 09:13:46 +00:00