The read-path test listed a hand-written movements path; it now also builds
paths with movement::request_path, so the test fails offline if the
allowlist and the builder drift apart (Pullfrog on #80).
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
- The session client's new read-path allowlist (#78) gains
`/insider/company/majorholder?`, the one endpoint movements reads; without
it every live movements query would fail before leaving the process. The
mock does not go through the client, so its tests could not show this.
- docs/ARCHITECTURE.md described one cache entry per page: a query's pages
are read and cached together, keyed by the page count, and a failed later
page returns the rows before it with `incomplete` (Pullfrog on #80).
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
From Pullfrog's review of #78:
- A dividend in another currency keeps 6 decimals (USD 0.0012 was rounded
to 0.0 in JSON and "USD 0" in the table); rupiah stays to the sen.
- `--kind stock_dividend` and `--kind reverse_split` work on the CLI too,
as they already did over MCP.
- The session client GETs only the prefixes in READ_PATHS, as
docs/STOCKBIT.md's rule 1 already said: any other path (orders, PIN,
withdrawals) fails before a request or token refresh leaves the process.
- `stocks calendar` and get_corporate_calendar also list rights trading,
tender offers and IPO offers still open in the range, so an offer that
began earlier shows while it can be acted on. A warrant's years-long
windows do not count. The 2026-11-03..12 golden gains DOOH, NAYZ and AUTO.
- `stocks calendar --help` names the ex-date fallback, and docs/STOCKBIT.md
shows the record and surface as shipped.
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
A block (UPSTREAMBLOCKED) was treated as a passing failure, so idx's browser
re-checked the same cookie every 5 s until the 15-minute timeout, warning
"does not retry a block ... retrying". A block holds for every session from
this network: the wait now ends with it at once.
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
idx-core reports what it works around (history falling back to yahoo when
Stockbit refuses the session, a stale cache served after a failed fetch, a
movements query cut short) as warnings on stderr. An agent never reads
idx-mcp's stderr, so it saw `source: "yahoo"` or `stale: true` with no reason.
- Each tool call collects its warnings with runtime::on_warning and returns
them as `warnings` on the answer or the error, when there are any.
- The stale-cache warning names the failure behind it.
- The mock's session API 403 is the real session API's block message; the
public page's block names IDX_PROVIDER=msn as well as `--provider msn`.
Goldens: error-intraday-blocked (json, table) carry the reworded block hint.
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
JSON `kind` now uses the `--kind` names, so an agent that copies a kind from
an answer cached or read before that change (`public_expose`,
`tender_offer`) would get INVALIDINPUT. Both are accepted, as hidden `--kind`
aliases and by the MCP tools.
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
A refused cookie in idx's browser was only a stderr warning, which nobody
reads under idx-mcp: stockbit_status showed `waiting` until the timeout, and
the timeout message did not mention the refusal.
- runtime::on_warning sends a thread's warnings to a sink as well as
stderr, until its guard drops.
- The background sign-in puts its warnings in `sign_in.warning`, cleared
once it signs in.
- The errors that end a browser wait (timeout, window closed, browser
exited) name a refusal it hit.
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
- Each page was cached on its own, so a multi-page answer could join
pages read at different times from a feed that gains rows at the top,
repeating some rows and missing others. A query's pages are now read
together and cached as one entry.
- A failed later page threw away the pages already read. They are now
returned with incomplete saying why, and not cached.
- The library did not cap pages at MAX_PAGES itself; it clamps now.
- The hint said to raise --pages even at 10; at 10 it says to narrow the
dates. -o jsonl, which streams rows only, gives the hint on stderr.
- A duplicate match arm in the CLI was dead; it is gone.
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
- JSON named kinds by their Rust names (public_expose, tender_offer,
stock_dividend, reverse_split) while --kind and MCP take expose, tender,
stock-dividend, reverse-split, so an output kind could not be passed
back. JSON now uses the input names; old cache entries still read, and
--kind also accepts underscores.
- {"dividend": null, "today": "..."} failed to parse and aborted the
whole command; a null list beside scalars now reads as none.
- An extreme --from panicked on the default 30 days; it is an error now.
The 366-day limit let 367 days through; it counts both ends now.
- An entitlement without a cum date sat on its ex date under a note that
calls DATE the cum date. The table marks it (ex) and says what that
means; JSON leaves cum_date out, as before, and the docs say so.
- A dividend's currency was ignored; one not in IDR now carries currency
and the table shows it instead of Rp.
- The live check no longer fails a quiet month's empty calendar.
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
Logged-in quotes passed Stockbit's session state through verbatim
(STATE_NAME_SESSION_2), while public-page quotes say open or close, so a
consumer checking for "open" broke after login. Quotes now use the two
words either way; the order book keeps the state verbatim.
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
auto history picks Stockbit when a session file exists, so an expired
session failed history and technical with AUTHERROR. auto now retries on
Yahoo with a warning; an explicit --history-provider stockbit still fails.
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
- login restored the previous session on any check error, even after the
check had rotated the pasted session: a timeout then left no working
session. It now keeps a rotated session unless Stockbit refused it.
- The browser sign-in marked a cookie refused on any error, so a 429 or a
timeout held a signed-in user until the wait ran out. Passing errors
are retried after a pause; only a refusal is final.
- The local page checked concurrent pastes on their own threads, so a
later one could overwrite the session the first had just rotated.
Pastes are now checked one at a time, and none after one is accepted.
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
stocks intraday never set quote.behind_session, so on a holiday or when
Stockbit lags, a past session's chart read as current. Mark it as quotes
do and print the same note under the header.
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
`idx ownership movements [SYMBOL]` and the MCP tool
`get_shareholder_movements` list reported changes in insiders' and major
holders' stakes, newest first, from Stockbit's
/insider/company/majorholder: IDX insider disclosures (any size, with the
trade price) and KSEI changes of holders at 5% and above (Stockbit's
estimated price). Stockbit filters by date range, holder id, action and
source; each page is 50 rows and one request, up to 10 pages, cached an
hour per page under every filter.
Rows keep exact signed share counts and the stake in basis points as
Stockbit rounds it; unknown action labels are kept as `other` rather than
dropped. JSON is {movements, more}; JSONL is one movement per line. The
command lives under `ownership` but needs no import, and `ownership`
commands now take --offline / --no-cache for it.
The per-symbol summary score is the next PR; docs/STOCKBIT.md records the
endpoint's parameters and action types as checked on 2026-11-03.
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
IDX's above-1% register as of 30 Oct 2026 (peng-2026-10-00029) inserts a
CONTROLLER_OR_AFFILIATE column after INVESTOR_CLASSIFICATION. The header
check rejected any column it did not know, so `ownership import` failed
with PARSEERROR and `releases`/`ticker` had nothing to show.
Accept that one column by name and skip it; any other unknown column is
still rejected, so the above-5% and investor-type workbooks stay out.
Storing the flag is a follow-up (ROADMAP).
The fixture is an 11-row excerpt of the real October workbook (disclaimer
rows, header, BBCA and ZYRX). The full file imports 7,149 rows across 961
tickers; live ownership smoke: 4 passed, 0 failed, no 403.
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
`idx stocks actions SYM [--kind ...]`: one stock's cash and stock
dividends, bonus shares, rights, warrants, splits and reverse splits,
meetings, public exposes and tender offers (`--kind ipo` adds its IPO),
newest first. `idx stocks calendar [--from] [--to] [--kind ...]`: the
market's actions dated in the range (default today and 30 days ahead),
oldest first. MCP: get_corporate_actions, get_corporate_calendar.
Stockbit's `/corpaction/{type}` serves one kind per request; each type has
its own field prefix and list key, so entitlements are read by field
suffix. An action's `date` is the cum date for entitlements, else the
event's first day. Dividends keep their decimals. One request per kind,
cached 6 hours. AUTHERROR without a session.
Fixtures are the 2026-11-03 recordings (market-wide ones trimmed to
October-December). Goldens added; help-stocks and completions-bash gain
the two subcommands, MCP tools.json the two tools (regenerated with
UPDATE_GOLDEN=1). The MCP docs counted 17 tools without stockbit_login;
now 20.
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
Observed in session on 2026-11-03:
- The regular board trades as STATE_NAME_SESSION_2, not the assumed
STATE_NAME_MARKET_OPEN, so session quotes hid bid/offer all day. Any
numbered STATE_NAME_SESSION_<n> now counts as trading; the lunch break
is STATE_NAME_BREAK.
- In session the order book's fbuy/fsell/fnet were the previous session's
(equal to IPOT's 2026-11-02 net for BBCA, BBRI and TLKM) and
has_foreign_bs was false. Foreign figures are kept only when that flag
is true.
Fixtures for the break, the trading state and an in-session book; the
table names the session ("market open (session 2)").
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
`stockbit::signin` is now the single sign-in flow. `idx stockbit login`
waits on it in the foreground; the `stockbit_login` MCP tool runs it on a
thread and `stockbit_status` reports `sign_in.state`, so an agent knows when
to stop polling. Auto uses a browser unless the shell is over SSH or has no
display, and falls back to the local page only when the browser cannot start.
The browser way opens Stockbit's sign-in page in a profile only idx uses and
reads the session cookie over DevTools. What a real sign-in showed:
- The cookie is host-only on stockbit.com, so deleting by the assumed
.stockbit.com removed nothing and left live tokens in the profile. It is
now deleted by its own domain and path, with every other tab closed first,
and read back to confirm.
- A signed-out page writes the cookie without tokens; it no longer counts.
- A profile belongs to one browser, so it records the executable that made it.
- On macOS the config directory depends on XDG_CONFIG_HOME, so a shell and an
app started from the Dock disagreed; both default locations are now checked.
- A thread does not outlive main, so idx-mcp closes a waiting browser when
stdin closes or on SIGTERM or Ctrl-C.
Progress goes to stderr, which keeps `-o json` to one document on stdout.
Google refused its own sign-in in this window on 2026-10-31, most likely over
the debugging flag; password sign-in works, and the docs and roadmap say so.
Claude-Session: https://claude.ai/code/session_01RqPPVaoFxLToGAXXBEoCtH
The login page listens on 127.0.0.1, so a browser on the Mac driving a VM
could never reach it. Over SSH, idx now picks port 7878, skips opening a
browser on the VM, and prints the exact ssh -L to run where the browser
is.
Browsers open idle preconnects; serving one connection at a time made the
page wait out the 3 s read timeout on each, which read as "can't
connect". Each connection now gets its own thread (capped) and a silent
one is dropped. Regression test; measured 10.1 s -> 0.08 s with three
idle sockets. Docs cover a browser on another machine (--port + ssh -L).
idx stockbit login in a terminal now serves a page on 127.0.0.1: a
bookmarklet that reads stockbit.com's session cookie in the user's own
browser and opens the page with it in the URL fragment, or a
read-the-clipboard button, with the manual paste as the fallback. A
per-run secret guards the path, the page is no-store with a strict CSP,
and it stops on success or after five minutes. --no-browser prints the
URL (--port pins it for ssh -L); piped stdin keeps the paste path.
A paste Stockbit refuses now restores the previous session instead of
losing it.
Opt-in, with the user's own Stockbit session from a pasted
credentialStorage cookie (or IDX_STOCKBIT_REFRESH_TOKEN):
- idx stockbit login/status/logout; session file 0600, tokens never shown
- rotating refresh under a cross-process lock; one refresh+retry on 401
- stocks orderbook: depth with lots and orders, OHLC, value, foreign flow
- --provider stockbit quotes from the session API when logged in
- --history-provider stockbit, and auto under the stockbit provider for
5d-1y daily windows
- MCP get_orderbook and stockbit_status (17 tools)
Endpoints, units and recorded responses from fzn0x/idx (MIT).
Read Stockbit's public symbol page (__NEXT_DATA__) with a plain ureq
request and an honest user agent: `stocks intraday` (today by the
minute, any provider), `--provider stockbit` quotes with the best
bid/offer only while the market is open, and MCP `get_intraday`.
- UPSTREAMBLOCKED error (exit 4) for a CDN 403; never retried
- requests paced 1 s apart across idx processes, 5 s snapshot cache
- global --provider flag (yahoo, msn, stockbit)
- roadmap: L1 ships from Stockbit; depth stays a login follow-on
Review findings on #71:
- The IDX session rule flagged non-IDX data: US screen rows on Monday
evening WIB, and JP/HK/SG rows on their own holidays. Only `.JK`
quotes and `--region id` screens are marked now.
- Nothing tested the marking call sites. A screen test checks IDX rows
online and offline, and US rows unmarked. A quote test reads a cache
entry in 0.3.3's format (no field) offline.
- The get_quotes description now says the flag also means the stock did
not trade, or an exchange holiday; screen_stocks mentions the field.
- latest_session is pub(crate).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
MSN can serve the previous session's close for hours after IDX closes;
on 2026-10-16 at 16:53 WIB it still returned the 15th. Quotes and
screener rows now carry `behind_session`, true when `as_of` is before
the latest weekday whose session has ended (16:15 WIB). It is computed
on return, not cached. The quote table notes such quotes, and the
get_quotes description explains the field.
Closes#50
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Classes are allowed per element type, not page-wide, and the table and
each footer element on the way to a total must carry exactly IPOT's
classes, so Bootstrap's visible-xs (hidden at desktop widths) on a total
or the table rejects the page (Pullfrog review).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Ancestor classes compare as sets (repeats removed too), and a page whose
echoed range can't be read is a parse error rather than a summary with
no dates, so the parser returns the exact range or nothing (Codex
re-check, round 13). The broker layer already rejected missing dates.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
From Codex's exhaustive audit of every allowed element, attribute and
returned value:
- class tokens split on ASCII whitespace as HTML does, so a no-break
space stays in the token (and fails the class allowlist);
- a date input's line breaks are dropped before parsing, as a text input
drops them;
- ancestor placement compares class token sets, not exact strings;
- the echoed range needs both date inputs, start then end in document
order, each with IPOT's id and name; otherwise no dates;
- a side with a blank broker code may show no figures.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
A header cell spanning columns would move the labels over the figures,
so header cells may carry only align. The broker kind comes from the
span's single class token as the browser splits it, so spacing in the
attribute no longer drops it (Codex re-check, round 11).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
An unknown code is only the reply that is exactly "Invalid stock code!",
not any page with that text (e.g. inside a script). The header row must
be IPOT's nine labels in order, so a page labelling the columns otherwise
is rejected rather than read by position (Codex re-check, round 10).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
The footer div must hold exactly IPOT's four text-only spans, in order and
each starting with its label, with nothing else shown between them, so a
total split from or across spans no longer reads as a smaller figure.
Pages over 64 KiB or 2,000 tags (IPOT's is about 8 KB and under 500) are
rejected before parsing, which bounds tree building on deep nesting
(Codex re-check, round 9).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Replace the list of hiding mechanisms with what the page is: every element
must be one IPOT uses, with only its attributes and class tokens, and with
input type, the footer div's style, colspan and button type pinned; the
table and date inputs must sit at IPOT's exact div paths; the table is
thead, tbody, tfoot; totals come from tfoot > tr > th > div > span only;
and a cell's span holds all the cell shows. A popover, an escaped inline
style, a hidden input, a total in details or a code beside its span now
reject the page, and the footer is walked directly (linear). Tests now
state the property: a page is rejected or read as shown, never as other
data (Codex re-check, round 8).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Inside details, dialog, noscript, object, select and the like they may
not show, so the table there rejects the page and such a date input is
no echoed date.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
On the html5ever DOM, a body cell must be a td (align only) holding text
or one span with a single text-* class holding text; footer totals must be
text-only spans, each label once; each date input must be unique; and a
template or an element hidden by markup (hidden, inline display:none or
visibility:hidden) anywhere rejects the page. So script or hidden text
inside a figure, a hidden total, a decoy span before a broker code, or a
duplicate date input can no longer be read as data, and footer text is
read once per span (Codex re-check, round 7).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Six rounds of the 0.4.0 review found crafted markup that the hand-written
reader read differently from HTML (cells in comments or attributes, stray
or spaced tags, a decoy date attribute or total, foreign content). Parse
the page with scraper (html5ever) instead: dates are the inputs' values,
rows are the single tbody's tr children with nine td each, totals are the
tfoot spans, all as a browser builds them. Rows that don't line up still
reject the page. scraper's CSS crates (selectors, cssparser) are MPL-2.0.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Comments are removed before parsing, and an unclosed or abrupt one
(<!-->, <!--->, --!>) or other <! / <? markup rejects the page; the page
is read only up to its first raw-text element (it ends with one empty
script). The echoed dates are read before the table and the totals after
it, cell text may not hold '>', and the row loop no longer rescans
trailing whitespace per row (Codex re-check, round 5).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Replace the tag scanner with a strict grammar: whole <tr> rows of nine
<td> cells, each holding text or one <span>, attributes as name="value"
without quotes or angle brackets in the value, and no character references
besides . Markup in that form reads the same in any HTML parser, so
a quoted '>', '</tr >', '<td-x' or '2' can no longer make the parser
see other cells than the page shows; such pages are rejected (Codex
re-check, round 4).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
One tbody start and end in the page, in any case, and inside it only tr,
td and span tags with no '<' before a tag closes. A stray </TbOdY> no
longer cuts the table short, and cells inside a comment or an attribute
are no longer read as brokers (Codex re-check).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
The body (one tbody, tags matched case-insensitively) must be whole rows:
each one <tr> holding nine opened and nine closed cells, ranked in order,
with nothing else between rows. A row with missing close tags no longer
merges into its neighbour, and an uppercase row is no longer skipped
(Codex re-check).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Every piece of the table body that holds a row must parse as nine cells;
ranks count the rows read, not the pieces. A rank-1 row without its cell
ends was skipped and ranks 2-10 still matched (Codex re-review).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
