Commit graph

5 commits

Author SHA1 Message Date
hermes
ec2e870c6a fix(cli): output contract, terminal-safe text, char-safe truncation, cache symlinks
From the full-repo Codex audit:
- Provider and KSEI text reached the terminal raw, so an escape sequence
  in a news title or insight could clear or rewrite the screen. Every
  table cell, insight line, and graph label now has control characters
  and bidirectional overrides replaced.
- Table rendering sliced strings by byte (a description at 200, a URL at
  72, an earnings period), panicking when a multi-byte character such as
  "’" or "é" fell on the cut. Truncate by character.
- In JSON mode, argument errors were clap's plain text and warnings were
  plain lines ahead of the JSON error. Argument errors are now the usual
  JSON error (INVALIDINPUT, exit 2) when JSON output is selected, and
  diagnostics are JSON lines; --help and --version are unchanged.
- `config` commands ignored the output format; they follow it like every
  other command (JSON when piped or with -o json, text on a terminal).
- Ownership argument mistakes (--limit 0, --format, missing or
  conflicting import sources, --family, an empty graph root) returned
  PARSEERROR / exit 1; they are INVALIDINPUT / exit 2 as documented.
- `cache clear` followed symlinked directories and deleted files outside
  the cache; it now removes links without following them.

Output change, intentional: goldens regenerated for the JSON argument
error and the five config commands, which run with stdout piped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-17 13:24:28 +00:00
hermes
1e551385d0 fix: close the remaining review findings (cross-process lock, partials)
Second Codex (gpt-6.1-sol, xhigh) pass over 60634d3, each item verified
before fixing:

- new: the sync recheck hashed an existing empty database before schema
  setup, so the first sync falsely reported an external change. Hash
  after setup; a published snapshot already has the schema and keeps
  its bytes.
- sync, import, and entity edits now hold a cross-process lock
  (<db>.lock, std File::lock), so a sync cannot replace the database
  under an import running in another process. Import validates and
  downloads before taking the lock, keeping its error order. MSRV
  1.85 -> 1.89 for File::lock.
- the row-count rebuild branch also never downgrades history.
- cache reads no longer delete corrupt entries (a concurrent writer may
  have just replaced them); the next put overwrites atomically.
- idx-mcp validates the changes limit before querying and runs each
  requested section or analysis once.
- XLSX parsing bounds total cell text, so one large shared string
  referenced from many cells cannot exhaust memory.
- PDF/archive percentages must be 0-100%; HHI uses saturating math.

CLI goldens unchanged. Live A/B vs 0.2.4: 16 passed, discover hits the
IDX 403 on both builds (exit 4 vs 1, intended).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 22:20:57 +00:00
hermes
60634d3aef fix: address final review (path traversal, sync races, MCP limits)
Findings from a Codex (gpt-6.1-sol, xhigh) review of v0.2.4..HEAD, each
verified in the code before fixing:

- high: ticker symbols reached cache file paths unchecked, so a symbol
  like `../../REPORT` read (and deleted, if unparseable) .json files
  outside the cache, reachable through MCP get_quotes. resolve_symbol now
  accepts only ticker characters, and cache keys are encoded as single
  safe path components.
- high: sync decided before downloading and replaced the database
  without rechecking; an import that landed meanwhile was erased. It now
  rechecks the local state before installing, and idx-mcp runs
  ownership_sync exclusively with ownership reads.
- replaces_sqlite_sha256 never downgrades history: the client requires
  the rebuild to keep at least the local release count, and the
  publisher lists a replaced build only then.
- cache writes are atomic (temp file + rename), so concurrent readers
  never see, and delete, a partial entry.
- idx-mcp caps symbols (20, deduplicated) and limits, and runs at most 4
  tool calls at once.
- each Yahoo client gets its own cookie jar, paired with its crumb.
- ZIP entries (XLSX, KSEI archives) are read with a 256 MiB bound;
  non-ASCII archive dates and huge percentages are parse errors instead
  of panics or overflow.
- agent-smoke --keep deletes every copied agent config, not only auth
  files.

CLI goldens unchanged; the MCP tool golden changes only to state the
new caps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 21:58:37 +00:00
hermes
86cb1b4b7c refactor(core): move stocks fetching, caching, and analysis into idx-core
Phase 3b of docs/DESIGN.md. `stocks` handlers are now argument parsing,
one idx-core call, and rendering.

- Idx (client.rs) is the entry point: Idx::new(CoreConfig) with
  market(), financial(), and analysis(). Live lookups return
  Fetched<T> { data, source, fetched_at, stale } and take a CachePolicy
  (Default | Offline | Refresh, from the --offline/--no-cache flags).
- One cached_fetch replaces the CLI's three cache flows; they differed
  only in whether a fresh entry is read via get or get_offline when
  offline, which yields the same data and diagnostics. Report-building
  errors (technical) still skip the stale-cache fallback.
- market.rs: quotes, history (history_source is resolved first so the
  CLI's fallback note keeps its position), and a typed screener
  (ScreenFilter/ScreenRegion with the same error messages).
- financial.rs: profile, statements, earnings, sentiment, insights,
  news, with StatementFilter/EarningsFilter.
- analysis: TechnicalReport moves to analysis::report; Analysis offers
  technical, growth, valuation, risk, fundamental, and compare.
- CoreConfig::load() owns provider/exchange/TTL settings from
  config.toml and IDX_*; the CLI's IdxConfig adds output and color.
  With two different invalid settings at once, the reported one may
  differ from before; every single-error message is unchanged.
- Cache gains fresh/any/offline accessors returning fetch metadata.

Unit tests moved with their code (core 157 -> 168, CLI 17 -> 7). All
143 goldens unchanged; live A/B against main: 16/16 stocks identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 09:41:31 +00:00
hermes
15947de51a refactor: split into idx-core library and idx-cli binary workspace
Phase 2 of docs/DESIGN.md, no behavior change:

- crates/idx-core (library): api, analysis, cache, config file handling
  and provider selection, curl_impersonate, error, ownership, runtime,
  plus the shared test fixtures. clap derives are behind an optional
  `clap` feature so the library does not require clap.
- crates/idx-cli (binary `idx`): cli/, output/, and IdxConfig (output
  format and color resolution). Package and binary names are unchanged.
- config.rs is split: file read/write and validation move to core; a
  CLI test keeps core's OUTPUT_VALUES in sync with OutputSetting.
- ownership::db::open_db/db_path drop their unused IdxConfig argument.
- One workspace version; shared dependency versions; no third-party
  version changes in Cargo.lock. LICENSE is symlinked into each crate.
- flake: workspace version, new source paths, and a per-crate package
  check (idx-cli is built from its tarball against packaged idx-core,
  since idx-core is not on crates.io yet). CI cache key, release path
  filter, and the golden whitespace-hook exclusion follow the new layout.

Parity: all 143 golden outputs match byte for byte without
regeneration; 157 core + 17 CLI unit tests (173 before plus the new
OUTPUT_VALUES sync test), 62 CLI tests pass. Live A/B against main:
16/16 stocks cases identical; ownership discover got IDX 403 on both
(exit 4 vs 1 is the phase 1 exit-code change).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 09:13:46 +00:00
Renamed from src/cache.rs (Browse further)