fix(ownership): report a Cloudflare challenge as its own error and result

When Cloudflare answers idx.co.id with a challenge, curl fails with only
`curl: (22)`, which exits 4 like any other HTTP failure, and every profile
is tried twice. A challenge needs a browser, so no profile can solve it. The
source is monthly, so the publisher only needs one run per month to get
through: treat a challenge as "retry tomorrow" and alert on stale data.

- Add IdxError::CloudflareChallenge (CLOUDFLARECHALLENGE, exit 6), detected
  from the cf-mitigated header. curl reports it on stderr through --write-out,
  because --fail drops the response.
- A URL with a challenge gets one round of profiles; other failures keep two.
  This also covers `stocks flow --source idx`, which now exits 6 on a challenge.
- -v logs which profile fetched each URL.
- Discovery keeps error types, so a challenge survives an announcement miss.
  A source answering with something unparseable still is not called a challenge.
- `ownership discover --require-data-page` fails when the data page cannot be
  read. Default discover is unchanged: it warns and lists older announcements.
- The publisher and builder use that flag, so a failed data page can no longer
  end in a false "up-to-date". A challenge ends in `RESULT: blocked stage=<stage>`,
  exit 6; every other failure stays `RESULT: FAILED`.
- The systemd example treats exit 6 as success and runs the freshness check, so
  the unit fails on FAILED or on stale data.
- live-smoke reports a challenge as CHALLENGED, a failure and never a pass.

Golden change: completions-bash.golden gains `--require-data-page` in the
discover options, one word on one line. It is the direct result of the new flag;
no other golden file changes.

Live check on 2026-11-05 against release/v0.5.0: 19 passed, 2 failed. Both
failures are real challenges (flow-idx, discover), each exit 6 with empty stdout.
The import, releases and ticker cases did not run, because discover did not pass.

Refs #83
This commit is contained in:
hermes 2026-11-05 08:54:58 +00:00
commit b236c7f9d9
23 changed files with 797 additions and 54 deletions

View file

@ -20,7 +20,7 @@ What an agent needs to know:
`{"error": true, "code": "...", "message": "..."}`.
- Exit codes say what to do next: `2` fix the input, `3` not found (or no
cached data offline), `4` provider unavailable, retry later, `5`
unsupported, `1` other.
unsupported, `6` Cloudflare challenge from idx.co.id, retry later, `1` other.
- Tickers are IDX codes (`BBCA`); `.JK` is optional. Several tickers can be
passed at once to `stocks quote` and `stocks compare`.
- Ownership queries need `idx ownership sync` once; after that they run

View file

@ -34,6 +34,7 @@ stdout is not a terminal, and with `--no-color`.
| `3` | not found, no data, or no cached data offline | check the ticker or name; go online |
| `4` | upstream unavailable | retry later |
| `5` | unsupported | use another command or provider |
| `6` | Cloudflare challenge (`CLOUDFLARECHALLENGE`), on commands that read idx.co.id | retry later, or from another network |
| `1` | anything else | read the message |
## Pipes

View file

@ -43,6 +43,7 @@ import IDX's reports directly:
```bash
idx ownership discover # find the latest report URLs on IDX
idx ownership discover --require-data-page # fail rather than list only older reports
idx ownership import --url <xlsx-or-pdf> # needs curl-impersonate
idx ownership import --file report.xlsx # or a local XLSX, PDF, ZIP or TXT
```

View file

@ -924,15 +924,16 @@ Discover the latest IDX-hosted ownership report URLs
Usage: idx ownership discover [OPTIONS]
Options:
--family <FAMILY> Report family to discover: above1 (default), all, above5, or investor-type [default: above1]
-o, --output <OUTPUT> Output format [default: auto = table on a terminal, json when piped] [possible values: auto, table, json, jsonl]
--limit <LIMIT> Maximum number of discovered report URLs to print [default: 6]
--no-color Disable colored output
-q, --quiet Suppress non-essential output
-v, --verbose... Increase verbosity (-v, -vv)
--offline Run without network requests
--no-cache Bypass local cache
-h, --help Print help
--family <FAMILY> Report family to discover: above1 (default), all, above5, or investor-type [default: above1]
-o, --output <OUTPUT> Output format [default: auto = table on a terminal, json when piped] [possible values: auto, table, json, jsonl]
--limit <LIMIT> Maximum number of discovered report URLs to print [default: 6]
--no-color Disable colored output
-q, --quiet Suppress non-essential output
--require-data-page Fail if the IDX data page cannot be read, instead of warning and listing older announcements. The newest above-1% reports are only on that page
-v, --verbose... Increase verbosity (-v, -vv)
--offline Run without network requests
--no-cache Bypass local cache
-h, --help Print help
Examples:
idx ownership discover --limit 1