0xrsydn/idx-cli
1
mirror of https://github.com/0xrsydn/idx-cli.git synced 2026-11-07 09:45:36 +00:00
Code Issues Projects Releases 1 Packages Wiki Activity

fix(core): parse IPOT pages with html5ever

Browse source 
Six rounds of the 0.4.0 review found crafted markup that the hand-written
reader read differently from HTML (cells in comments or attributes, stray
or spaced tags, a decoy date attribute or total, foreign content). Parse
the page with scraper (html5ever) instead: dates are the inputs' values,
rows are the single tbody's tr children with nine td each, totals are the
tfoot spans, all as a browser builds them. Rows that don't line up still
reject the page. scraper's CSS crates (selectors, cssparser) are MPL-2.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
This commit is contained in:
hermes 2026-10-18 12:54:34 +00:00
commit 79b15d51b9
4 changed files with 517 additions and 321 deletions
Download patch file Download diff file Expand all files Collapse all files

1
crates/idx-core/Cargo.toml
View file

@ -34,3 +34,4 @@ rusqlite = { workspace = true, optional = true }
quick-xml = { version = "0.37", optional = true }
sha2 = { workspace = true, optional = true }
zip = { workspace = true, optional = true }
scraper = { version = "0.27", default-features = false }