fix(ownership): sync snapshots over 10 MiB; refuse oversized publishes

On 2026-09-25 a snapshot built with --history 5 (15.3 MB) was
published, and `idx ownership sync` failed for every client:
"the response body is larger than request limit: 10485760" (ureq's
default body cap). It was republished by hand with --history 2 (8.2 MB).

- snapshot download: raise the ureq body limit to 1 GiB; the file is
  still verified against the manifest's size and SHA-256. The PDF/XLSX
  report downloads get a 256 MiB limit for the same reason.
- CLI test: sync a >10 MiB snapshot over HTTP (fails without the fix
  with the exact production error).
- build-latest-ownership-snapshot.sh: --max-snapshot-bytes (default
  10485760, the limit of released clients <= v0.2.3); an oversized
  snapshot fails the build, so the published one stays untouched.
- publish-ownership-snapshot.sh passes --max-snapshot-bytes through;
  publisher tests cover the failure and that the release is unchanged.
- docs: size table and guidance.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017a5Cv5Z1NzdYKgKnTiAF4E
This commit is contained in:
hermes 2026-09-25 22:22:48 +00:00
commit de58d04da9
7 changed files with 156 additions and 5 deletions

View file

@ -16,6 +16,11 @@ pub const SNAPSHOT_MANIFEST_ENV: &str = "IDX_OWNERSHIP_SNAPSHOT_MANIFEST";
pub const SNAPSHOT_MANIFEST_SCHEMA_VERSION: u32 = 1;
pub const DEFAULT_SNAPSHOT_MANIFEST_URL: &str = "https://github.com/0xrsydn/idx-cli/releases/download/ownership-snapshot-current/ownership-snapshot-manifest.json";
/// ureq caps response bodies at 10 MiB by default. Snapshots with several
/// months of history exceed that (history 3 is ~10.5 MB), and the download is
/// verified against the manifest's size and SHA-256 anyway, so allow up to 1 GiB.
pub(crate) const SNAPSHOT_DOWNLOAD_LIMIT_BYTES: u64 = 1024 * 1024 * 1024;
const USER_AGENT: &str = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36";
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
@ -654,8 +659,10 @@ fn read_bytes(source: &str, context: &str) -> Result<Vec<u8>, IdxError> {
)
.call()
.map_err(|e| IdxError::Http(format!("failed to fetch {context}: {e}")))?;
let mut body = response.into_body();
return body
return response
.into_body()
.with_config()
.limit(SNAPSHOT_DOWNLOAD_LIMIT_BYTES)
.read_to_vec()
.map_err(|e| IdxError::Http(format!("failed reading {context} body: {e}")));
}