From the Codex re-gate of the 0.3.3 fixes:
- `changes` broke ties between same-date releases by row id, but the
ticker, entity, and graph queries and the snapshot builder did not, so
two files imported in the same second could resolve differently per
query. Order by as_of_date, imported_at, then id everywhere.
- search::fts_search and search::rebuild_fts were public in a published
crate; keep them as deprecated wrappers.
- Invalid --source and --by are INVALIDINPUT, like other argument errors.
- The prefix-ranking test could not tell prefix from exact-alias ranking;
add one where only prefix vs substring differs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
From a full-repo Codex audit, each reproduced on the synced ownership DB:
- `changes` paired an investor's accounts across releases instead of
comparing totals: KREN's KRESNA PRIMA INVEST went 9.44% -> 13.45% (a
second 4.01% account) but was reported as decreased 944 -> 401 bps.
Sum a holder's accounts per ticker, and read one release per date (the
latest imported, as other queries do), so a corrected file supersedes
the original instead of mixing with it.
- `changes` / get_ownership_changes accepted any dates: `from: ""`
reported all 7,154 holders as new, unknown dates returned no changes.
Dates must be YYYY-MM-DD imported releases with `from` before `to`.
- get_investor_holdings / `entity` with an empty name matched every
investor and returned an arbitrary one ("AMY"); `%` and `_` acted as
LIKE wildcards. Reject empty names; match wildcards literally.
- Alias search never worked: the FTS table declared an `aliases` column
its content table lacks (`no such column: T.aliases`), so searches
fell back to canonical names and a raw KSEI name such as
"PT ASTRA INTERNATIONAL TBK" found nothing. Search names and aliases
directly, ranked exact name, exact alias, prefix, then shorter names.
- `graph` enumerated every path (exponential in depth) and its cycle
check matched substrings, dropping entity:1 when entity:10 was on the
path. Traverse breadth-first with an exact visited set.
- Name normalization removed leading PT tokens from the front of a Vec,
quadratic in hostile names; narrow a slice instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
Final Codex (gpt-6.1-sol, xhigh) pass over c65982f confirmed every
earlier item fixed and found:
- high: readers skipped the lock on any error, so a lock file owned by
another user (who can still sync) let a reader run unlocked. Readers
now open an existing lock file read-only for the shared lock, and read
unlocked only when no lock file exists and none can be created.
- low: sync resolved the symlinked path for the install target and the
lock separately; it now resolves once and passes that path to the
lock callback, so both always name the same database.
Verified live: a reader with a read-only lock file waits for an
exclusive holder; install waits for readers; a symlinked path keeps its
link. CLI goldens unchanged; live A/B vs 0.2.4: 15 passed, screen hit a
transient MSN 408 on the old build only (retries pass), discover hits
the IDX 403 on both (exit 4 vs 1, intended).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fourth Codex (gpt-6.1-sol, xhigh) pass over 5e0831d, each item verified
before fixing:
- sync inspects the local database under the shared lock, so another
sync cannot swap it under that connection.
- symlinks are resolved by reading the link (db::resolve_db_path), not
canonicalize, so a symlinked path finds the same lock and install
target even while the real file is missing (fresh install, mid-swap)
or the link dangles.
- readers go ahead unlocked when no lock file can be created; nothing
can replace the database there. (A read-only data directory still
fails in SQLite's WAL setup, as in 0.2.4.)
- XLSX shared strings are capped at 2,000,000 entries.
Verified live: an install waits for a reader's shared lock; a symlinked
db_path waits on the real path's lock and stays a symlink. CLI goldens
unchanged; live A/B vs 0.2.4: 16 passed, discover hits the IDX 403 on
both builds (exit 4 vs 1, intended).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Third Codex (gpt-6.1-sol, xhigh) pass over 1e55138, each item verified
before fixing:
- readers now hold the ownership lock shared for the life of their
connection, so a snapshot swap (exclusive) never happens under an
open SQLite connection whose WAL cleanup could hit a later import.
- the lock file name uses the resolved database path, so a symlinked
path shares the real database's lock; sync also installs at the
resolved path, keeping a symlinked path a symlink (it used to be
overwritten with a regular file).
- sync downloads without the lock and takes it only to recheck and
install; snapshot downloads have connect (15 s) and total (10 min)
timeouts. idx-mcp drops its in-process RwLock: idx-core's lock covers
it across processes.
- XLSX parsing caps rows at 1,000,000 and checks a shared string's size
before copying it.
- the temp snapshot's -wal/-shm files are removed instead of left in the
data directory.
Remaining, by design: `changes` loads all rows of two releases from the
local database before filtering; its size is bounded by the snapshot.
CLI goldens unchanged. Live A/B vs 0.2.4: 16 passed; discover hits the
IDX 403 on both builds (exit 4 vs 1, intended).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Second Codex (gpt-6.1-sol, xhigh) pass over 60634d3, each item verified
before fixing:
- new: the sync recheck hashed an existing empty database before schema
setup, so the first sync falsely reported an external change. Hash
after setup; a published snapshot already has the schema and keeps
its bytes.
- sync, import, and entity edits now hold a cross-process lock
(<db>.lock, std File::lock), so a sync cannot replace the database
under an import running in another process. Import validates and
downloads before taking the lock, keeping its error order. MSRV
1.85 -> 1.89 for File::lock.
- the row-count rebuild branch also never downgrades history.
- cache reads no longer delete corrupt entries (a concurrent writer may
have just replaced them); the next put overwrites atomically.
- idx-mcp validates the changes limit before querying and runs each
requested section or analysis once.
- XLSX parsing bounds total cell text, so one large shared string
referenced from many cells cannot exhaust memory.
- PDF/archive percentages must be 0-100%; HHI uses saturating math.
CLI goldens unchanged. Live A/B vs 0.2.4: 16 passed, discover hits the
IDX 403 on both builds (exit 4 vs 1, intended).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Phase 4 of docs/DESIGN.md. Eleven grouped tools (quotes, history, screen,
company, analysis, compare, shareholders, investor holdings, search,
ownership changes, status); ownership_sync is listed only with
--allow-sync. Live results keep the Fetched envelope, errors use the CLI's
{error, code, message} body, and investor tools on an empty database
point at the sync path.
idx-core gains Period/Interval FromStr, Investor::status, a ticker
filter on changes, and Display for ErrorCode (now used by the CLI too).
CLI output and goldens are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Phase 3c of docs/DESIGN.md. `ownership` handlers are now argument
parsing, one idx-core call, and rendering.
- investor.rs: Investor (also Idx::investor()) with releases, ticker,
entity, search, cross_holders, concentration, flow, changes, graph,
unresolved, map_alias, merge_entities, discover, sync, and import.
Import orchestration (input resolution, download, SHA-256 dedup,
PDF/XLSX/archive parsing, entity resolution, release write) moves
from the CLI unchanged.
- Typed inputs: ImportSource, HolderSource, ConcentrationRank, and
parse_report_family, with the same error messages.
- CoreConfig gains an ownership_db override; db::open_db_at opens an
explicit path. idx-cli no longer depends on rusqlite, sha2, or ureq
at runtime.
- Behavior note: an invalid `ticker --source` or `concentration --by`
is now rejected before the database file is opened (and created);
the error and exit code are unchanged.
- Fix: building idx-cli without the ownership feature no longer warns
about the ownership-only `blue` style.
All 143 goldens unchanged; 170 core + 7 CLI unit tests and 62 CLI tests
pass; clippy clean for default, release, testing, and no-default
feature sets. Live A/B against main: 16/16 stocks identical; ownership
discover got IDX 403 on both (exit 4 vs 1 is the phase 1 change).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
