Commit graph

9 commits

Author SHA1 Message Date
hermes
6cfceafe04 fix(core): read IPOT cells and footer only in the shape IPOT writes
On the html5ever DOM, a body cell must be a td (align only) holding text
or one span with a single text-* class holding text; footer totals must be
text-only spans, each label once; each date input must be unique; and a
template or an element hidden by markup (hidden, inline display:none or
visibility:hidden) anywhere rejects the page. So script or hidden text
inside a figure, a hidden total, a decoy span before a broker code, or a
duplicate date input can no longer be read as data, and footer text is
read once per span (Codex re-check, round 7).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 13:09:13 +00:00
hermes
79b15d51b9 fix(core): parse IPOT pages with html5ever
Six rounds of the 0.4.0 review found crafted markup that the hand-written
reader read differently from HTML (cells in comments or attributes, stray
or spaced tags, a decoy date attribute or total, foreign content). Parse
the page with scraper (html5ever) instead: dates are the inputs' values,
rows are the single tbody's tr children with nine td each, totals are the
tfoot spans, all as a browser builds them. Rows that don't line up still
reject the page. scraper's CSS crates (selectors, cssparser) are MPL-2.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 12:54:34 +00:00
hermes
1c6ec6eb28 fix(core): read the IPOT page as a browser would around its table
Comments are removed before parsing, and an unclosed or abrupt one
(<!-->, <!--->, --!>) or other <! / <? markup rejects the page; the page
is read only up to its first raw-text element (it ends with one empty
script). The echoed dates are read before the table and the totals after
it, cell text may not hold '>', and the row loop no longer rescans
trailing whitespace per row (Codex re-check, round 5).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 12:34:16 +00:00
hermes
cecfbab235 fix(core): accept only the exact markup IPOT serves in its table body
Replace the tag scanner with a strict grammar: whole <tr> rows of nine
<td> cells, each holding text or one <span>, attributes as name="value"
without quotes or angle brackets in the value, and no character references
besides &nbsp;. Markup in that form reads the same in any HTML parser, so
a quoted '>', '</tr >', '<td-x' or '&#50;' can no longer make the parser
see other cells than the page shows; such pages are rejected (Codex
re-check, round 4).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 12:24:44 +00:00
hermes
186d9557d6 fix(core): the IPOT table body holds plain row markup only
One tbody start and end in the page, in any case, and inside it only tr,
td and span tags with no '<' before a tag closes. A stray </TbOdY> no
longer cuts the table short, and cells inside a comment or an attribute
are no longer read as brokers (Codex re-check).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 12:15:40 +00:00
hermes
30afcc109d fix(core): read the IPOT table body as whole rows only
The body (one tbody, tags matched case-insensitively) must be whole rows:
each one <tr> holding nine opened and nine closed cells, ranked in order,
with nothing else between rows. A row with missing close tags no longer
merges into its neighbour, and an uppercase row is no longer skipped
(Codex re-check).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 12:05:13 +00:00
hermes
106590d151 fix(core): a row with no parsed cells rejects the page too
Every piece of the table body that holds a row must parse as nine cells;
ranks count the rows read, not the pieces. A rank-1 row without its cell
ends was skipped and ranks 2-10 still matched (Codex re-review).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 11:56:25 +00:00
hermes
d82f7c6be9 fix(core): reject broker and trading pages that would misreport a session
From the 0.4.0 release review (Codex):
- IPOT pages with a short row, a rank out of order or a broker after a
  blank row are parse errors; skipping a row shifted later brokers up a
  rank and could flip a bandar reading (-2425 bps to -66 bps).
- A tbody end before its start is a parse error, not a panic.
- An IPOT page that echoes other dates than requested is rejected before
  it is cached, so one session's figures never pass as another's.
- An IDX trading summary with rows for other days only is an error, not
  an empty session cached for 30 days.
- Empty sessions count toward a flow's fetched_at and stale.
- The bandar imbalance uses checked i128 arithmetic: past i128 there is
  no reading instead of a wrapped one.
- The bandar table prints "No trades." only when no broker traded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 11:48:00 +00:00
hermes
2c93c73b6a feat(core): broker summaries from IPOT
Adds a BrokerSummaryProvider capability served by Indo Premier's public
broker summary page: the top 10 buyers and sellers of a stock by lots over
a date range, with totals. No login or key.

- api/ipot: pure parser (decimal scaling of `1.6 M`-style figures), a
  client that retries the server's spurious 404s, and a fixture mock.
- broker: Broker::summary/latest over the cache (7 days for closed ranges,
  5 minutes for today), range checks, and IDX member names bundled from
  GetBrokerCodeList.
- foreign_net_value is null for filtered summaries: the source does not
  apply the filters to it.

The endpoint's behavior was first mapped by fzn0x/idx (MIT).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 04:28:38 +00:00