Commit graph

17 commits

Author SHA1 Message Date
hermes
7d55be80c5 fix(core): bind IPOT class tokens to the elements that carry them
Classes are allowed per element type, not page-wide, and the table and
each footer element on the way to a total must carry exactly IPOT's
classes, so Bootstrap's visible-xs (hidden at desktop widths) on a total
or the table rejects the page (Pullfrog review).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 14:35:02 +00:00
hermes
3c5f9bb8a4 fix(core): an IPOT page without a readable date range is rejected
Ancestor classes compare as sets (repeats removed too), and a page whose
echoed range can't be read is a parse error rather than a summary with
no dates, so the parser returns the exact range or nothing (Codex
re-check, round 13). The broker layer already rejected missing dates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 14:25:45 +00:00
hermes
bf77073877 fix(core): close the round-12 audit of the IPOT parser
From Codex's exhaustive audit of every allowed element, attribute and
returned value:
- class tokens split on ASCII whitespace as HTML does, so a no-break
  space stays in the token (and fails the class allowlist);
- a date input's line breaks are dropped before parsing, as a text input
  drops them;
- ancestor placement compares class token sets, not exact strings;
- the echoed range needs both date inputs, start then end in document
  order, each with IPOT's id and name; otherwise no dates;
- a side with a blank broker code may show no figures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 14:15:51 +00:00
hermes
d15fc4833a fix(core): header cells take only align; broker kind from the class token
A header cell spanning columns would move the labels over the figures,
so header cells may carry only align. The broker kind comes from the
span's single class token as the browser splits it, so spacing in the
attribute no longer drops it (Codex re-check, round 11).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 13:58:59 +00:00
hermes
828d16c92e fix(core): match IPOT's unknown-code reply exactly; check column labels
An unknown code is only the reply that is exactly "Invalid stock code!",
not any page with that text (e.g. inside a script). The header row must
be IPOT's nine labels in order, so a page labelling the columns otherwise
is rejected rather than read by position (Codex re-check, round 10).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 13:49:15 +00:00
hermes
ee4fe4b852 fix(core): read IPOT's four footer spans exactly; bound the page size
The footer div must hold exactly IPOT's four text-only spans, in order and
each starting with its label, with nothing else shown between them, so a
total split from or across spans no longer reads as a smaller figure.
Pages over 64 KiB or 2,000 tags (IPOT's is about 8 KB and under 500) are
rejected before parsing, which bounds tree building on deep nesting
(Codex re-check, round 9).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 13:40:01 +00:00
hermes
63c4d9342f fix(core): accept only IPOT's own markup, in its own places
Replace the list of hiding mechanisms with what the page is: every element
must be one IPOT uses, with only its attributes and class tokens, and with
input type, the footer div's style, colspan and button type pinned; the
table and date inputs must sit at IPOT's exact div paths; the table is
thead, tbody, tfoot; totals come from tfoot > tr > th > div > span only;
and a cell's span holds all the cell shows. A popover, an escaped inline
style, a hidden input, a total in details or a code beside its span now
reject the page, and the footer is walked directly (linear). Tests now
state the property: a page is rejected or read as shown, never as other
data (Codex re-check, round 8).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 13:30:08 +00:00
hermes
723da4a5a6 fix(core): the IPOT table and date inputs must sit in plain divs
Inside details, dialog, noscript, object, select and the like they may
not show, so the table there rejects the page and such a date input is
no echoed date.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 13:18:30 +00:00
hermes
6cfceafe04 fix(core): read IPOT cells and footer only in the shape IPOT writes
On the html5ever DOM, a body cell must be a td (align only) holding text
or one span with a single text-* class holding text; footer totals must be
text-only spans, each label once; each date input must be unique; and a
template or an element hidden by markup (hidden, inline display:none or
visibility:hidden) anywhere rejects the page. So script or hidden text
inside a figure, a hidden total, a decoy span before a broker code, or a
duplicate date input can no longer be read as data, and footer text is
read once per span (Codex re-check, round 7).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 13:09:13 +00:00
hermes
79b15d51b9 fix(core): parse IPOT pages with html5ever
Six rounds of the 0.4.0 review found crafted markup that the hand-written
reader read differently from HTML (cells in comments or attributes, stray
or spaced tags, a decoy date attribute or total, foreign content). Parse
the page with scraper (html5ever) instead: dates are the inputs' values,
rows are the single tbody's tr children with nine td each, totals are the
tfoot spans, all as a browser builds them. Rows that don't line up still
reject the page. scraper's CSS crates (selectors, cssparser) are MPL-2.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 12:54:34 +00:00
hermes
1c6ec6eb28 fix(core): read the IPOT page as a browser would around its table
Comments are removed before parsing, and an unclosed or abrupt one
(<!-->, <!--->, --!>) or other <! / <? markup rejects the page; the page
is read only up to its first raw-text element (it ends with one empty
script). The echoed dates are read before the table and the totals after
it, cell text may not hold '>', and the row loop no longer rescans
trailing whitespace per row (Codex re-check, round 5).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 12:34:16 +00:00
hermes
cecfbab235 fix(core): accept only the exact markup IPOT serves in its table body
Replace the tag scanner with a strict grammar: whole <tr> rows of nine
<td> cells, each holding text or one <span>, attributes as name="value"
without quotes or angle brackets in the value, and no character references
besides &nbsp;. Markup in that form reads the same in any HTML parser, so
a quoted '>', '</tr >', '<td-x' or '&#50;' can no longer make the parser
see other cells than the page shows; such pages are rejected (Codex
re-check, round 4).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 12:24:44 +00:00
hermes
186d9557d6 fix(core): the IPOT table body holds plain row markup only
One tbody start and end in the page, in any case, and inside it only tr,
td and span tags with no '<' before a tag closes. A stray </TbOdY> no
longer cuts the table short, and cells inside a comment or an attribute
are no longer read as brokers (Codex re-check).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 12:15:40 +00:00
hermes
30afcc109d fix(core): read the IPOT table body as whole rows only
The body (one tbody, tags matched case-insensitively) must be whole rows:
each one <tr> holding nine opened and nine closed cells, ranked in order,
with nothing else between rows. A row with missing close tags no longer
merges into its neighbour, and an uppercase row is no longer skipped
(Codex re-check).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 12:05:13 +00:00
hermes
106590d151 fix(core): a row with no parsed cells rejects the page too
Every piece of the table body that holds a row must parse as nine cells;
ranks count the rows read, not the pieces. A rank-1 row without its cell
ends was skipped and ranks 2-10 still matched (Codex re-review).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 11:56:25 +00:00
hermes
d82f7c6be9 fix(core): reject broker and trading pages that would misreport a session
From the 0.4.0 release review (Codex):
- IPOT pages with a short row, a rank out of order or a broker after a
  blank row are parse errors; skipping a row shifted later brokers up a
  rank and could flip a bandar reading (-2425 bps to -66 bps).
- A tbody end before its start is a parse error, not a panic.
- An IPOT page that echoes other dates than requested is rejected before
  it is cached, so one session's figures never pass as another's.
- An IDX trading summary with rows for other days only is an error, not
  an empty session cached for 30 days.
- Empty sessions count toward a flow's fetched_at and stale.
- The bandar imbalance uses checked i128 arithmetic: past i128 there is
  no reading instead of a wrapped one.
- The bandar table prints "No trades." only when no broker traded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 11:48:00 +00:00
hermes
2c93c73b6a feat(core): broker summaries from IPOT
Adds a BrokerSummaryProvider capability served by Indo Premier's public
broker summary page: the top 10 buyers and sellers of a stock by lots over
a date range, with totals. No login or key.

- api/ipot: pure parser (decimal scaling of `1.6 M`-style figures), a
  client that retries the server's spurious 404s, and a fixture mock.
- broker: Broker::summary/latest over the cache (7 days for closed ranges,
  5 minutes for today), range checks, and IDX member names bundled from
  GetBrokerCodeList.
- foreign_net_value is null for filtered summaries: the source does not
  apply the filters to it.

The endpoint's behavior was first mapped by fzn0x/idx (MIT).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 04:28:38 +00:00