mirror of
https://github.com/0xrsydn/idx-cli.git
synced 2026-11-07 09:45:36 +00:00
fix(core): parse IPOT pages with html5ever
Six rounds of the 0.4.0 review found crafted markup that the hand-written reader read differently from HTML (cells in comments or attributes, stray or spaced tags, a decoy date attribute or total, foreign content). Parse the page with scraper (html5ever) instead: dates are the inputs' values, rows are the single tbody's tr children with nine td each, totals are the tfoot spans, all as a browser builds them. Rows that don't line up still reject the page. scraper's CSS crates (selectors, cssparser) are MPL-2.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
This commit is contained in:
parent
1c6ec6eb28
commit
79b15d51b9
4 changed files with 517 additions and 321 deletions
|
|
@ -92,7 +92,12 @@ WIB as the fallback.
|
|||
takes `code`, `start`/`end` (`MM/DD/YYYY`), `fd` (`all|F|D`) and `board`
|
||||
(`all|RG|TN|NG`). It needs no login and answers an HTML fragment: the top 10
|
||||
buyers and top 10 sellers ranked by lots (gross, so a broker can be on both
|
||||
sides), plus range totals. `api/ipot/parse.rs` parses it purely.
|
||||
sides), plus range totals. `api/ipot/parse.rs` parses it purely, with
|
||||
html5ever (`scraper`), so it reads the cells, date inputs and footer a
|
||||
browser would build: a hand-written reader kept disagreeing with HTML on
|
||||
crafted markup in the 0.4.0 review. A table whose rows don't line up
|
||||
(missing cells, ranks out of order, a broker after a blank row, a second
|
||||
body) rejects the page, and the dates it echoes must match the request.
|
||||
- Figures of a million and more arrive abbreviated (`1.6 M`); they are scaled
|
||||
in decimal and the summary's `rounded` flag says so. `F. NVal` ignores the
|
||||
investor and board filters, so `foreign_net_value` is null for a summary of
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue