idx stockbit login in a terminal now serves a page on 127.0.0.1: a
bookmarklet that reads stockbit.com's session cookie in the user's own
browser and opens the page with it in the URL fragment, or a
read-the-clipboard button, with the manual paste as the fallback. A
per-run secret guards the path, the page is no-store with a strict CSP,
and it stops on success or after five minutes. --no-browser prints the
URL (--port pins it for ssh -L); piped stdin keeps the paste path.
A paste Stockbit refuses now restores the previous session instead of
losing it.
Opt-in, with the user's own Stockbit session from a pasted
credentialStorage cookie (or IDX_STOCKBIT_REFRESH_TOKEN):
- idx stockbit login/status/logout; session file 0600, tokens never shown
- rotating refresh under a cross-process lock; one refresh+retry on 401
- stocks orderbook: depth with lots and orders, OHLC, value, foreign flow
- --provider stockbit quotes from the session API when logged in
- --history-provider stockbit, and auto under the stockbit provider for
5d-1y daily windows
- MCP get_orderbook and stockbit_status (17 tools)
Endpoints, units and recorded responses from fzn0x/idx (MIT).
Read Stockbit's public symbol page (__NEXT_DATA__) with a plain ureq
request and an honest user agent: `stocks intraday` (today by the
minute, any provider), `--provider stockbit` quotes with the best
bid/offer only while the market is open, and MCP `get_intraday`.
- UPSTREAMBLOCKED error (exit 4) for a CDN 403; never retried
- requests paced 1 s apart across idx processes, 5 s snapshot cache
- global --provider flag (yahoo, msn, stockbit)
- roadmap: L1 ships from Stockbit; depth stays a login follow-on
Review findings on #71:
- The IDX session rule flagged non-IDX data: US screen rows on Monday
evening WIB, and JP/HK/SG rows on their own holidays. Only `.JK`
quotes and `--region id` screens are marked now.
- Nothing tested the marking call sites. A screen test checks IDX rows
online and offline, and US rows unmarked. A quote test reads a cache
entry in 0.3.3's format (no field) offline.
- The get_quotes description now says the flag also means the stock did
not trade, or an exchange holiday; screen_stocks mentions the field.
- latest_session is pub(crate).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
MSN can serve the previous session's close for hours after IDX closes;
on 2026-10-16 at 16:53 WIB it still returned the 15th. Quotes and
screener rows now carry `behind_session`, true when `as_of` is before
the latest weekday whose session has ended (16:15 WIB). It is computed
on return, not cached. The quote table notes such quotes, and the
get_quotes description explains the field.
Closes#50
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Classes are allowed per element type, not page-wide, and the table and
each footer element on the way to a total must carry exactly IPOT's
classes, so Bootstrap's visible-xs (hidden at desktop widths) on a total
or the table rejects the page (Pullfrog review).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Ancestor classes compare as sets (repeats removed too), and a page whose
echoed range can't be read is a parse error rather than a summary with
no dates, so the parser returns the exact range or nothing (Codex
re-check, round 13). The broker layer already rejected missing dates.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
From Codex's exhaustive audit of every allowed element, attribute and
returned value:
- class tokens split on ASCII whitespace as HTML does, so a no-break
space stays in the token (and fails the class allowlist);
- a date input's line breaks are dropped before parsing, as a text input
drops them;
- ancestor placement compares class token sets, not exact strings;
- the echoed range needs both date inputs, start then end in document
order, each with IPOT's id and name; otherwise no dates;
- a side with a blank broker code may show no figures.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
A header cell spanning columns would move the labels over the figures,
so header cells may carry only align. The broker kind comes from the
span's single class token as the browser splits it, so spacing in the
attribute no longer drops it (Codex re-check, round 11).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
An unknown code is only the reply that is exactly "Invalid stock code!",
not any page with that text (e.g. inside a script). The header row must
be IPOT's nine labels in order, so a page labelling the columns otherwise
is rejected rather than read by position (Codex re-check, round 10).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
The footer div must hold exactly IPOT's four text-only spans, in order and
each starting with its label, with nothing else shown between them, so a
total split from or across spans no longer reads as a smaller figure.
Pages over 64 KiB or 2,000 tags (IPOT's is about 8 KB and under 500) are
rejected before parsing, which bounds tree building on deep nesting
(Codex re-check, round 9).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Replace the list of hiding mechanisms with what the page is: every element
must be one IPOT uses, with only its attributes and class tokens, and with
input type, the footer div's style, colspan and button type pinned; the
table and date inputs must sit at IPOT's exact div paths; the table is
thead, tbody, tfoot; totals come from tfoot > tr > th > div > span only;
and a cell's span holds all the cell shows. A popover, an escaped inline
style, a hidden input, a total in details or a code beside its span now
reject the page, and the footer is walked directly (linear). Tests now
state the property: a page is rejected or read as shown, never as other
data (Codex re-check, round 8).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Inside details, dialog, noscript, object, select and the like they may
not show, so the table there rejects the page and such a date input is
no echoed date.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
On the html5ever DOM, a body cell must be a td (align only) holding text
or one span with a single text-* class holding text; footer totals must be
text-only spans, each label once; each date input must be unique; and a
template or an element hidden by markup (hidden, inline display:none or
visibility:hidden) anywhere rejects the page. So script or hidden text
inside a figure, a hidden total, a decoy span before a broker code, or a
duplicate date input can no longer be read as data, and footer text is
read once per span (Codex re-check, round 7).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Six rounds of the 0.4.0 review found crafted markup that the hand-written
reader read differently from HTML (cells in comments or attributes, stray
or spaced tags, a decoy date attribute or total, foreign content). Parse
the page with scraper (html5ever) instead: dates are the inputs' values,
rows are the single tbody's tr children with nine td each, totals are the
tfoot spans, all as a browser builds them. Rows that don't line up still
reject the page. scraper's CSS crates (selectors, cssparser) are MPL-2.0.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Comments are removed before parsing, and an unclosed or abrupt one
(<!-->, <!--->, --!>) or other <! / <? markup rejects the page; the page
is read only up to its first raw-text element (it ends with one empty
script). The echoed dates are read before the table and the totals after
it, cell text may not hold '>', and the row loop no longer rescans
trailing whitespace per row (Codex re-check, round 5).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Replace the tag scanner with a strict grammar: whole <tr> rows of nine
<td> cells, each holding text or one <span>, attributes as name="value"
without quotes or angle brackets in the value, and no character references
besides . Markup in that form reads the same in any HTML parser, so
a quoted '>', '</tr >', '<td-x' or '2' can no longer make the parser
see other cells than the page shows; such pages are rejected (Codex
re-check, round 4).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
One tbody start and end in the page, in any case, and inside it only tr,
td and span tags with no '<' before a tag closes. A stray </TbOdY> no
longer cuts the table short, and cells inside a comment or an attribute
are no longer read as brokers (Codex re-check).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
The body (one tbody, tags matched case-insensitively) must be whole rows:
each one <tr> holding nine opened and nine closed cells, ranked in order,
with nothing else between rows. A row with missing close tags no longer
merges into its neighbour, and an uppercase row is no longer skipped
(Codex re-check).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Every piece of the table body that holds a row must parse as nine cells;
ranks count the rows read, not the pieces. A rank-1 row without its cell
ends was skipped and ranks 2-10 still matched (Codex re-review).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
From the 0.4.0 release review (Codex):
- IPOT pages with a short row, a rank out of order or a broker after a
blank row are parse errors; skipping a row shifted later brokers up a
rank and could flip a bandar reading (-2425 bps to -66 bps).
- A tbody end before its start is a parse error, not a panic.
- An IPOT page that echoes other dates than requested is rejected before
it is cached, so one session's figures never pass as another's.
- An IDX trading summary with rows for other days only is an error, not
an empty session cached for 30 days.
- Empty sessions count toward a flow's fetched_at and stale.
- The bandar imbalance uses checked i128 arithmetic: past i128 there is
no reading instead of a wrapped one.
- The bandar table prints "No trades." only when no broker traded.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
`idx stocks flow` now reads IPOT by default, which needs nothing installed:
per session, the all-investor broker summary gives the value traded and
the foreign one gives foreign buying (its footer), the foreign net and
selling (buying less the net), across all boards, through the broker
summary cache. IPOT's foreign net matched IDX's estimate within 0.3 B on
BBCA 2026-10-12 to 16.
`--source idx` (and `source` on get_foreign_flow) keeps IDX's trading
summary: exact share counts, close and the non-regular share, behind
curl-impersonate and Cloudflare.
- ForeignFlowDay gains foreign_buy_value, foreign_sell_value and
foreign_share_bps for both sources (estimated at the session average
for IDX); ForeignFlow gains source, the value totals and rounded.
- The table lays out each source's figures; the live smoke runs both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
The cache stores normalized summaries, so one written before buy_value and
sell_value existed read back without them: bandar gave null readings (the
table said "No trades.") for a week, and an investor-filtered entry lacked
its totals. Summaries now cache under ipot-broker-v2, so an old-shape entry
is never served; offline it is a cache miss, online it is fetched again. A
CLI test seeds an old entry and checks both, in JSON and table output.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
IPOT's investor filter splits each broker row by client domicile, but its
footer then gives only that side's buying (BBCA 2026-10-16: foreign 1.7 T,
domestic 4.9 T of 6.6 T). The foreign net gives the selling on all boards:
foreign 1.7 + 2.3 = 4.0 T (IDX: about 3.95 T regular), domestic
4.9 - 2.3 = 2.6 T.
- BrokerSummary gains buy_value and sell_value: the total for all
investors, the footer and the derived selling for one side (selling is
null with a board filter, where the net covers other boards).
- Bandar weighs the top N buyers' share of the value bought against the
top N sellers' share of the value sold, so `--investor foreign|domestic`
reads that side alone; for all investors the readings are unchanged.
One side on one board is refused, as there is no selling to weigh.
- Tables print what the side bought and sold; fixtures for foreign and
domestic on all boards were recorded from IPOT.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
IDX's Value is regular-board turnover, so NonRegularValue / Value gave
non-regular over regular (BBRI 2026-10-14: 50.91%) instead of the share of
all trading (33.74%). non_regular_bps now divides by regular plus
non-regular value, with a regression test on that fixture. The docs now say
volume, value and the foreign figures are the regular board: IPOT's
regular total matches Value, and in a week of every stock no foreign figure
exceeded regular volume, even with non-regular volume above twice it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
`idx stocks flow SYMBOL` shows what foreign investors bought and sold of a
stock, session by session, from IDX's own daily trading summary: foreign
buy, sell and net volume, net value at the session's average price, and
the share of value traded outside the regular board, with range totals.
Latest 10 sessions by default (--days, at most 60) or --from/--to. The MCP
server exposes it as get_foreign_flow.
- api/idx: pure parser for GetStockSummary (rows dated another session
are dropped) and a provider over curl-impersonate's profile fallback
with a 30 s limit; Cloudflare passes only some fingerprints.
- flow: each session is cached whole by date, so other stocks' flows for
the same sessions need no request. Weekends are never requested.
- Idx::idx_code is shared by broker and flow for the .JK-only check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
- Basis points truncate toward zero instead of flooring, so a reading of
-4.9952% stays neutral like its mirror image and -14.9952% stays
distribution; tests cover both signs at both thresholds through
bandar_tier. Goldens move by the rounding (BBCA top 5: -2426 -> -2425).
- With one investor side selected the summary has no total to weigh
brokers against, so bandar and analyze_bandar reject it (INVALIDINPUT)
instead of reading every tier as missing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
`idx stocks bandar SYMBOL...` reads the broker summary: for the top 1, 3,
5 and 10 brokers per side, buy value minus sell value in basis points of
value traded, with a signal from big accumulation to big distribution on
the headline tier (--top, default 5). Several symbols give a screen ranked
by that figure (at most 100, one request each), which pipes from
`stocks screen`. The MCP server exposes it as analyze_bandar (20 symbols).
- analysis/bandar.rs is pure and versioned (`top_concentration_v1`):
integer basis points, no figure missing counted as zero, buy/sell VWAP
from exact averages, and the brokers on both sides listed because their
gross figures overstate their direction.
- Thresholds (±5%/±15%) follow fzn0x/idx (MIT); docs call them heuristics.
- Invalid input fails a screen at once; other errors fail one symbol and
are warned about on stderr.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
`idx stocks broker SYMBOL` shows the top 10 buyers and sellers of a stock
by lots for the latest session with trades, one `--date`, or a
`--from`/`--to` range, filtered by `--investor` and `--board`. The MCP
server exposes the same as get_broker_summary.
- Range and filter parsing lives in idx-core (BrokerRange::from_args,
FromStr for the filters), so both frontends accept the same input.
- Without dates, the latest session is found by walking back over
weekdays; offline, uncached days are skipped.
- Table output lists foreign and state-owned brokers, and says when the
source rounded figures.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
With fd=F or fd=D, IPOT's footer covers only that side's buying: BBCA's
foreign/regular summary for 2026-10-16 gives T. Val 1.6 T while its top
seller alone sold 2.0 T. Passing it through as the range total gave
consumers a denominator smaller than one listed broker, so total_value,
total_lots and average_price are now null for an investor-filtered summary.
The page-to-summary step is a pure function, tested on both fixtures.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Adds a BrokerSummaryProvider capability served by Indo Premier's public
broker summary page: the top 10 buyers and sellers of a stock by lots over
a date range, with totals. No login or key.
- api/ipot: pure parser (decimal scaling of `1.6 M`-style figures), a
client that retries the server's spurious 404s, and a fixture mock.
- broker: Broker::summary/latest over the cache (7 days for closed ranges,
5 minutes for today), range checks, and IDX member names bundled from
GetBrokerCodeList.
- foreign_net_value is null for filtered summaries: the source does not
apply the filters to it.
The endpoint's behavior was first mapped by fzn0x/idx (MIT).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
From the full-repo Codex audit:
- The overall fundamental rating was "healthy" whenever growth was
growing and valuation and risk were merely not "expensive" / not "high
risk", so with no valuation or risk data at all (partial MSN
fundamentals) it still read "healthy". It now needs valuation
undervalued or fairly valued and risk low or moderate.
- Risk recognized only heavy leverage or a weak current ratio as high
risk; negative equity fell through to "moderate risk", and a known
adverse signal with the other metric missing became "incomplete data".
Any known adverse signal now makes it high risk.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
From the narrow Codex confirmation: provider, KSEI, and manifest text
still reached the terminal through error messages (a Yahoo error
description, a manifest's `kind`), idx-core diagnostics, the ownership
heading's ticker code and Bing date, the technical heading and
comparison symbol, and printed paths. Sanitize where every error
(emit_error) and every diagnostic (runtime) is printed instead of at each
caller, plus the remaining direct prints. terminal_safe moves to
idx_core::text so the runtime can use it; the CLI re-exports it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
From the full-repo Codex audit:
- Provider and KSEI text reached the terminal raw, so an escape sequence
in a news title or insight could clear or rewrite the screen. Every
table cell, insight line, and graph label now has control characters
and bidirectional overrides replaced.
- Table rendering sliced strings by byte (a description at 200, a URL at
72, an earnings period), panicking when a multi-byte character such as
"’" or "é" fell on the cut. Truncate by character.
- In JSON mode, argument errors were clap's plain text and warnings were
plain lines ahead of the JSON error. Argument errors are now the usual
JSON error (INVALIDINPUT, exit 2) when JSON output is selected, and
diagnostics are JSON lines; --help and --version are unchanged.
- `config` commands ignored the output format; they follow it like every
other command (JSON when piped or with -o json, text on a terminal).
- Ownership argument mistakes (--limit 0, --format, missing or
conflicting import sources, --family, an empty graph root) returned
PARSEERROR / exit 1; they are INVALIDINPUT / exit 2 as documented.
- `cache clear` followed symlinked directories and deleted files outside
the cache; it now removes links without following them.
Output change, intentional: goldens regenerated for the JSON argument
error and the five config commands, which run with stdout piped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Reverts a360d0b. A separate process group let an overflow kill the curl
behind a profile script, but Ctrl+C then reached only idx and left a
stalled transfer running (Pullfrog review on #55). Cancellation is the
common path; on overflow, curl still ends on its next write to the
closed pipe, and the error returns at once either way.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Run curl in its own process group and kill the group, so the curl
behind a curl_* wrapper script ends at once instead of at its next
write or --max-time (Pullfrog review on #55).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
The curl_* profiles run curl without exec, so killing the wrapper left
curl holding stderr, and joining stderr waited up to --max-time. Close
stdout and return; curl ends on its next write (Pullfrog review on #55).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Command::output() buffered the whole response before the size check, and
curl before 8.20 does not apply --max-filesize to decoded bodies, so a
small compressed response could exhaust memory. Read stdout up to the
limit, then kill and reap curl (Pullfrog review on #55).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
From the Codex re-gate of the 0.3.3 fixes: the price sanity check also
applied to MSN's priceChange, so a falling stock without a previous close
got change: null. Bound a change by its size instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
From the full-repo Codex audit:
- MSN and Yahoo clients set connect and body timeouts but none for the
response headers, so a server that stalled before answering hung the
call; four such calls take every MCP call slot. Add a 30 s overall
deadline. Every curl-impersonate run gets --max-time 600 and a 256 MiB
--max-filesize, and output above that is rejected (curl cannot cap a
response without a length); the Yahoo cookie fetch gets --max-time.
- Ownership commands blocked on the database lock without limit, so a
long sync or import, or a stuck process, held MCP call slots
indefinitely. Wait up to 30 s, then fail with a "busy, try again" error.
- A price of 1e20 saturated `as i64` and the change subtraction then
overflowed. Prices that are not finite, negative, or above 1e12 IDR are
treated as missing, and a quote's own price as an error.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
A pre-0.3.3 install interrupted between its renames left only a backup.
Restoring it inside the install step came after the decision, so an
older snapshot replaced a newer database without --force, and a failed
download left no database (Pullfrog review on #54).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
From the Codex re-gate of the 0.3.3 fixes:
- Imports recorded row_count as the rows parsed, but INSERT OR IGNORE
drops a file's repeated ticker/investor rows, so the count could exceed
what is stored, and the new stored-row validation would then reject a
snapshot the publisher built from such a file. Record the rows stored.
- A local manifest over the 1 MiB limit was read truncated rather than
rejected; read one byte more and fail when it does not fit.
- The backup-recovery test also passed on the old code; it now fails an
install after the leftover backup was found and checks the old data is
back.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
From the full-repo Codex audit:
- Install moved the database to a backup, then renamed the snapshot in;
a crash between the two left no database, and the next sync deleted the
backup first. Replace it with a single rename (old file or new, never
neither), and restore a backup an older version left behind.
- Validation trusted the release's row_count metadata, so a snapshot with
an empty or missing holdings table passed and replaced good data. Also
count the rows stored for the latest release.
- A manifest fetched over HTTP(S) could name a local download_url such as
/dev/zero, read without a size limit. Remote manifests must give an
HTTP(S) artifact; local reads take regular files only, within the size
limit, and manifests are capped at 1 MiB.
- A same-date build with a different row count replaced the local file
even when it did not list it in replaces_sqlite_sha256, so a replayed
older build could undo a corrected one. Same-date rebuilds now go only
through replaces_sqlite_sha256, which the publisher fills.
Verified against the published snapshot: fresh install, no-op re-sync,
and forced reinstall all succeed with no leftover files.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
A KSEI file can carry rows from several report dates; its release is
dated by the latest. Filtering on the release alone pulled January rows
into a February comparison (Pullfrog review on #53).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
From the Codex re-gate of the 0.3.3 fixes:
- `changes` broke ties between same-date releases by row id, but the
ticker, entity, and graph queries and the snapshot builder did not, so
two files imported in the same second could resolve differently per
query. Order by as_of_date, imported_at, then id everywhere.
- search::fts_search and search::rebuild_fts were public in a published
crate; keep them as deprecated wrappers.
- Invalid --source and --by are INVALIDINPUT, like other argument errors.
- The prefix-ranking test could not tell prefix from exact-alias ranking;
add one where only prefix vs substring differs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
From a full-repo Codex audit, each reproduced on the synced ownership DB:
- `changes` paired an investor's accounts across releases instead of
comparing totals: KREN's KRESNA PRIMA INVEST went 9.44% -> 13.45% (a
second 4.01% account) but was reported as decreased 944 -> 401 bps.
Sum a holder's accounts per ticker, and read one release per date (the
latest imported, as other queries do), so a corrected file supersedes
the original instead of mixing with it.
- `changes` / get_ownership_changes accepted any dates: `from: ""`
reported all 7,154 holders as new, unknown dates returned no changes.
Dates must be YYYY-MM-DD imported releases with `from` before `to`.
- get_investor_holdings / `entity` with an empty name matched every
investor and returned an arbitrary one ("AMY"); `%` and `_` acted as
LIKE wildcards. Reject empty names; match wildcards literally.
- Alias search never worked: the FTS table declared an `aliases` column
its content table lacks (`no such column: T.aliases`), so searches
fell back to canonical names and a raw KSEI name such as
"PT ASTRA INTERNATIONAL TBK" found nothing. Search names and aliases
directly, ranked exact name, exact alias, prefix, then shorter names.
- `graph` enumerated every path (exponential in depth) and its cycle
check matched substrings, dropping entity:1 when entity:10 was on the
path. Traverse breadth-first with an exact visited set.
- Name normalization removed leading PT tokens from the front of a Vec,
quadratic in hostile names; narrow a slice instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Every zip 2.x release from 2.4.2 on is yanked; crates.io warned on each
0.3.2 publish. zip 9 returns entry names as a Result (it validates their
encoding): skip undecodable names, which can be neither the archive's TXT
payload nor an XLSX worksheet. zip 9 needs Rust 1.88, within our 1.89.
Closes#51.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
String order put 10:00:00Z before the newer 10:00:00.500Z and mixed
time-zone offsets; with a small --limit the newest item could drop out.
Compare parsed RFC 3339 instants; missing or unparseable dates sort last.
From a Codex review.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
MSN's feed for a stock is often mostly general market news: of 10 items,
ADRO, BREN and CUAN had none about the company, BBRI one, yet they were
presented as the company's news. Fetch at least 50 items, label each
`relevance: company` (tagged by MSN with the stock's instrument id, or
naming the ticker or the company) or `market`, and return company items
first, newest first, trimmed to --limit.
Company names come from the quote's short and display names, best
effort: MSN tags Telkom articles with its NYSE ADR, not TLKM, so names
catch what tags miss. Classification stays a pure function in parse_news.
Output change, intentional: news JSON gains `relevance`, the table an
ABOUT column (goldens regenerated), and the get_company tool description
explains the field (tools.json and the website tools page regenerated).
News cache keys change so entries without the field are not reused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Final Codex (gpt-6.1-sol, xhigh) pass over c65982f confirmed every
earlier item fixed and found:
- high: readers skipped the lock on any error, so a lock file owned by
another user (who can still sync) let a reader run unlocked. Readers
now open an existing lock file read-only for the shared lock, and read
unlocked only when no lock file exists and none can be created.
- low: sync resolved the symlinked path for the install target and the
lock separately; it now resolves once and passes that path to the
lock callback, so both always name the same database.
Verified live: a reader with a read-only lock file waits for an
exclusive holder; install waits for readers; a symlinked path keeps its
link. CLI goldens unchanged; live A/B vs 0.2.4: 15 passed, screen hit a
transient MSN 408 on the old build only (retries pass), discover hits
the IDX 403 on both (exit 4 vs 1, intended).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
