- Why not a browser per user: a full engine per install and fetch, each
user facing the challenge, IDX seeing thousands of scrapers; one
scheduled publisher serving static files instead (diagram). Hosting
left open (GitHub release, Cloudflare Pages/R2, a Worker).
- The IDX challenge is JavaScript-based: curl fails from a second
datacenter (greencloud-vps, Singapore) too, so IP is not what curl
lacks.
- Other sources measured: Google News RSS covers companies MSN misses
(BREN 75/100 titles naming the ticker in 7 days), plain HTTP; Yahoo
search answers 429.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
MSN has no company news for many tickers; IDX's disclosures cover every
issuer. idx.co.id is behind a Cloudflare challenge that plain curl and
curl-impersonate fail from the maintainer host, while the local Camofox
browser gets through. Record the working endpoints (per-issuer and
all-issuer disclosures, exchange news), their shapes and quirks, the
options, and a proposed design: a maintainer publisher writes static JSON
to a GitHub release, as the ownership snapshot does, and idx-core merges
it into news.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
