Commit graph

86 commits

Author SHA1 Message Date
4dee7fe4dd Merge branch 'feat/stockbit-login' into feat/stockbit-corpaction
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 19:22:25 +07:00
362d314118 fix(stockbit): tell a polling agent when Stockbit refuses the sign-in
A refused cookie in idx's browser was only a stderr warning, which nobody
reads under idx-mcp: stockbit_status showed `waiting` until the timeout, and
the timeout message did not mention the refusal.

- runtime::on_warning sends a thread's warnings to a sink as well as
  stderr, until its guard drops.
- The background sign-in puts its warnings in `sign_in.warning`, cleared
  once it signs in.
- The errors that end a browser wait (timeout, window closed, browser
  exited) name a refusal it hit.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 19:22:14 +07:00
6ad57dd997 Merge branch 'feat/stockbit-login' into feat/stockbit-corpaction
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 17:31:35 +07:00
81a3fe7266 Merge branch 'feat/stockbit' into feat/stockbit-login
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 17:24:55 +07:00
6f8d3c071a fix(intraday): mark a chart from before the latest session
stocks intraday never set quote.behind_session, so on a holiday or when
Stockbit lags, a past session's chart read as current. Mark it as quotes
do and print the same note under the header.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 17:24:42 +07:00
1b72c3ed6b docs(smoke): the Stockbit session cases and why they share the real config
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-03 14:38:04 +07:00
43ff4aa433 feat: corporate actions from the Stockbit session (stocks actions, stocks calendar)
`idx stocks actions SYM [--kind ...]`: one stock's cash and stock
dividends, bonus shares, rights, warrants, splits and reverse splits,
meetings, public exposes and tender offers (`--kind ipo` adds its IPO),
newest first. `idx stocks calendar [--from] [--to] [--kind ...]`: the
market's actions dated in the range (default today and 30 days ahead),
oldest first. MCP: get_corporate_actions, get_corporate_calendar.

Stockbit's `/corpaction/{type}` serves one kind per request; each type has
its own field prefix and list key, so entitlements are read by field
suffix. An action's `date` is the cum date for entitlements, else the
event's first day. Dividends keep their decimals. One request per kind,
cached 6 hours. AUTHERROR without a session.

Fixtures are the 2026-11-03 recordings (market-wide ones trimmed to
October-December). Goldens added; help-stocks and completions-bash gain
the two subcommands, MCP tools.json the two tools (regenerated with
UPDATE_GOLDEN=1). The MCP docs counted 17 tools without stockbit_login;
now 20.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-03 14:35:20 +07:00
8a1b29043c docs: Stockbit session endpoints and the design of features on them
docs/STOCKBIT.md: every market-data endpoint in Stockbit's web client,
probed with a session on 2026-11-03 (corporate actions, shareholder
movements, composition, market detector, analyst consensus, hourly charts,
market states), the read-only rules all session features follow, and the
design of corporate actions and shareholder movements. Linked from
AGENTS.md and ROADMAP.md.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-03 14:04:32 +07:00
32f476bae7 fix(stockbit): bid/offer while trading; drop stale foreign flow
Observed in session on 2026-11-03:

- The regular board trades as STATE_NAME_SESSION_2, not the assumed
  STATE_NAME_MARKET_OPEN, so session quotes hid bid/offer all day. Any
  numbered STATE_NAME_SESSION_<n> now counts as trading; the lunch break
  is STATE_NAME_BREAK.
- In session the order book's fbuy/fsell/fnet were the previous session's
  (equal to IPOT's 2026-11-02 net for BBCA, BBRI and TLKM) and
  has_foreign_bs was false. Foreign figures are kept only when that flag
  is true.

Fixtures for the break, the trading state and an in-session book; the
table names the session ("market open (session 2)").

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-03 14:02:03 +07:00
ca4d3d09c2 feat(stockbit): one browser sign-in in idx-core for the CLI and MCP
`stockbit::signin` is now the single sign-in flow. `idx stockbit login`
waits on it in the foreground; the `stockbit_login` MCP tool runs it on a
thread and `stockbit_status` reports `sign_in.state`, so an agent knows when
to stop polling. Auto uses a browser unless the shell is over SSH or has no
display, and falls back to the local page only when the browser cannot start.

The browser way opens Stockbit's sign-in page in a profile only idx uses and
reads the session cookie over DevTools. What a real sign-in showed:

- The cookie is host-only on stockbit.com, so deleting by the assumed
  .stockbit.com removed nothing and left live tokens in the profile. It is
  now deleted by its own domain and path, with every other tab closed first,
  and read back to confirm.
- A signed-out page writes the cookie without tokens; it no longer counts.
- A profile belongs to one browser, so it records the executable that made it.
- On macOS the config directory depends on XDG_CONFIG_HOME, so a shell and an
  app started from the Dock disagreed; both default locations are now checked.
- A thread does not outlive main, so idx-mcp closes a waiting browser when
  stdin closes or on SIGTERM or Ctrl-C.

Progress goes to stderr, which keeps `-o json` to one document on stdout.

Google refused its own sign-in in this window on 2026-10-31, most likely over
the debugging flag; password sign-in works, and the docs and roadmap say so.

Claude-Session: https://claude.ai/code/session_01RqPPVaoFxLToGAXXBEoCtH
2026-10-31 23:58:21 +07:00
hermes
12fdc8c8e8 feat(stockbit): connect a session from a local browser page
idx stockbit login in a terminal now serves a page on 127.0.0.1: a
bookmarklet that reads stockbit.com's session cookie in the user's own
browser and opens the page with it in the URL fragment, or a
read-the-clipboard button, with the manual paste as the fallback. A
per-run secret guards the path, the page is no-store with a strict CSP,
and it stops on success or after five minutes. --no-browser prints the
URL (--port pins it for ssh -L); piped stdin keeps the paste path.

A paste Stockbit refuses now restores the previous session instead of
losing it.
2026-10-23 12:29:01 +00:00
hermes
1d4ab65853 docs: Stockbit order-book sizes are shares, checked against a live session 2026-10-23 10:09:43 +00:00
hermes
b5ae9b98de feat: Stockbit login for the order book, live quotes and daily history
Opt-in, with the user's own Stockbit session from a pasted
credentialStorage cookie (or IDX_STOCKBIT_REFRESH_TOKEN):

- idx stockbit login/status/logout; session file 0600, tokens never shown
- rotating refresh under a cross-process lock; one refresh+retry on 401
- stocks orderbook: depth with lots and orders, OHLC, value, foreign flow
- --provider stockbit quotes from the session API when logged in
- --history-provider stockbit, and auto under the stockbit provider for
  5d-1y daily windows
- MCP get_orderbook and stockbit_status (17 tools)

Endpoints, units and recorded responses from fzn0x/idx (MIT).
2026-10-23 08:35:57 +00:00
hermes
74e5bb0536 feat: login-free Stockbit intraday and quote provider
Read Stockbit's public symbol page (__NEXT_DATA__) with a plain ureq
request and an honest user agent: `stocks intraday` (today by the
minute, any provider), `--provider stockbit` quotes with the best
bid/offer only while the market is open, and MCP `get_intraday`.

- UPSTREAMBLOCKED error (exit 4) for a CDN 403; never retried
- requests paced 1 s apart across idx processes, 5 s snapshot cache
- global --provider flag (yahoo, msn, stockbit)
- roadmap: L1 ships from Stockbit; depth stays a login follow-on
2026-10-23 04:58:32 +00:00
hermes
160014c504 chore: release 0.4.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-19 10:50:06 +00:00
hermes
834e47b336 fix: mark only IDX quotes behind the session; test the wiring
Review findings on #71:
- The IDX session rule flagged non-IDX data: US screen rows on Monday
  evening WIB, and JP/HK/SG rows on their own holidays. Only `.JK`
  quotes and `--region id` screens are marked now.
- Nothing tested the marking call sites. A screen test checks IDX rows
  online and offline, and US rows unmarked. A quote test reads a cache
  entry in 0.3.3's format (no field) offline.
- The get_quotes description now says the flag also means the stock did
  not trade, or an exchange holiday; screen_stocks mentions the field.
- latest_session is pub(crate).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-19 10:26:09 +00:00
hermes
a0a2cac7cf feat: flag quotes behind the latest IDX session (#50)
MSN can serve the previous session's close for hours after IDX closes;
on 2026-10-16 at 16:53 WIB it still returned the 15th. Quotes and
screener rows now carry `behind_session`, true when `as_of` is before
the latest weekday whose session has ended (16:15 WIB). It is computed
on return, not cached. The quote table notes such quotes, and the
get_quotes description explains the field.

Closes #50

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 17:38:48 +00:00
hermes
79b15d51b9 fix(core): parse IPOT pages with html5ever
Six rounds of the 0.4.0 review found crafted markup that the hand-written
reader read differently from HTML (cells in comments or attributes, stray
or spaced tags, a decoy date attribute or total, foreign content). Parse
the page with scraper (html5ever) instead: dates are the inputs' values,
rows are the single tbody's tr children with nine td each, totals are the
tfoot spans, all as a browser builds them. Rows that don't line up still
reject the page. scraper's CSS crates (selectors, cssparser) are MPL-2.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 12:54:34 +00:00
hermes
8b507d78e2 feat: foreign flow from IPOT by default (--source ipot|idx)
`idx stocks flow` now reads IPOT by default, which needs nothing installed:
per session, the all-investor broker summary gives the value traded and
the foreign one gives foreign buying (its footer), the foreign net and
selling (buying less the net), across all boards, through the broker
summary cache. IPOT's foreign net matched IDX's estimate within 0.3 B on
BBCA 2026-10-12 to 16.

`--source idx` (and `source` on get_foreign_flow) keeps IDX's trading
summary: exact share counts, close and the non-regular share, behind
curl-impersonate and Cloudflare.

- ForeignFlowDay gains foreign_buy_value, foreign_sell_value and
  foreign_share_bps for both sources (estimated at the session average
  for IDX); ForeignFlow gains source, the value totals and rounded.
- The table lays out each source's figures; the live smoke runs both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 06:40:22 +00:00
hermes
5f64319157 feat: bandar and broker totals for one investor side
IPOT's investor filter splits each broker row by client domicile, but its
footer then gives only that side's buying (BBCA 2026-10-16: foreign 1.7 T,
domestic 4.9 T of 6.6 T). The foreign net gives the selling on all boards:
foreign 1.7 + 2.3 = 4.0 T (IDX: about 3.95 T regular), domestic
4.9 - 2.3 = 2.6 T.

- BrokerSummary gains buy_value and sell_value: the total for all
  investors, the footer and the derived selling for one side (selling is
  null with a board filter, where the net covers other boards).
- Bandar weighs the top N buyers' share of the value bought against the
  top N sellers' share of the value sold, so `--investor foreign|domestic`
  reads that side alone; for all investors the readings are unchanged.
  One side on one board is refused, as there is no selling to weigh.
- Tables print what the side bought and sold; fixtures for foreign and
  domestic on all boards were recorded from IPOT.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 06:08:02 +00:00
hermes
83783684cb fix(core): non-regular share is of all trading
IDX's Value is regular-board turnover, so NonRegularValue / Value gave
non-regular over regular (BBRI 2026-10-14: 50.91%) instead of the share of
all trading (33.74%). non_regular_bps now divides by regular plus
non-regular value, with a regression test on that fixture. The docs now say
volume, value and the foreign figures are the regular board: IPOT's
regular total matches Value, and in a week of every stock no foreign figure
exceeded regular volume, even with non-regular volume above twice it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 05:41:55 +00:00
hermes
29f143fa1b feat: foreign flow (idx stocks flow, get_foreign_flow)
`idx stocks flow SYMBOL` shows what foreign investors bought and sold of a
stock, session by session, from IDX's own daily trading summary: foreign
buy, sell and net volume, net value at the session's average price, and
the share of value traded outside the regular board, with range totals.
Latest 10 sessions by default (--days, at most 60) or --from/--to. The MCP
server exposes it as get_foreign_flow.

- api/idx: pure parser for GetStockSummary (rows dated another session
  are dropped) and a provider over curl-impersonate's profile fallback
  with a 30 s limit; Cloudflare passes only some fingerprints.
- flow: each session is cached whole by date, so other stocks' flows for
  the same sessions need no request. Weekends are never requested.
- Idx::idx_code is shared by broker and flow for the .JK-only check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 05:39:12 +00:00
hermes
970da26a0e feat: bandarmology (idx stocks bandar, analyze_bandar)
`idx stocks bandar SYMBOL...` reads the broker summary: for the top 1, 3,
5 and 10 brokers per side, buy value minus sell value in basis points of
value traded, with a signal from big accumulation to big distribution on
the headline tier (--top, default 5). Several symbols give a screen ranked
by that figure (at most 100, one request each), which pipes from
`stocks screen`. The MCP server exposes it as analyze_bandar (20 symbols).

- analysis/bandar.rs is pure and versioned (`top_concentration_v1`):
  integer basis points, no figure missing counted as zero, buy/sell VWAP
  from exact averages, and the brokers on both sides listed because their
  gross figures overstate their direction.
- Thresholds (±5%/±15%) follow fzn0x/idx (MIT); docs call them heuristics.
- Invalid input fails a screen at once; other errors fail one symbol and
  are warned about on stderr.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 05:34:37 +00:00
hermes
7c8151a3a6 fix(core): no broker totals for one investor side
With fd=F or fd=D, IPOT's footer covers only that side's buying: BBCA's
foreign/regular summary for 2026-10-16 gives T. Val 1.6 T while its top
seller alone sold 2.0 T. Passing it through as the range total gave
consumers a denominator smaller than one listed broker, so total_value,
total_lots and average_price are now null for an investor-filtered summary.
The page-to-summary step is a pure function, tested on both fixtures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 05:32:29 +00:00
hermes
2c93c73b6a feat(core): broker summaries from IPOT
Adds a BrokerSummaryProvider capability served by Indo Premier's public
broker summary page: the top 10 buyers and sellers of a stock by lots over
a date range, with totals. No login or key.

- api/ipot: pure parser (decimal scaling of `1.6 M`-style figures), a
  client that retries the server's spurious 404s, and a fixture mock.
- broker: Broker::summary/latest over the cache (7 days for closed ranges,
  5 minutes for today), range checks, and IDX member names bundled from
  GetBrokerCodeList.
- foreign_net_value is null for filtered summaries: the source does not
  apply the filters to it.

The endpoint's behavior was first mapped by fzn0x/idx (MIT).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 04:28:38 +00:00
hermes
ebdba13a41 chore: release 0.3.3
Patch release for the full-repo audit of 0.3.2: ownership query
correctness (changes, alias search, graph, investor lookup), snapshot
sync safety, request and lock time limits, the CLI output contract and
terminal-safe output, fundamental rating logic, publisher and installer
robustness, and zip off a yanked release (#51).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 02:16:18 +00:00
hermes
ec2e870c6a fix(cli): output contract, terminal-safe text, char-safe truncation, cache symlinks
From the full-repo Codex audit:
- Provider and KSEI text reached the terminal raw, so an escape sequence
  in a news title or insight could clear or rewrite the screen. Every
  table cell, insight line, and graph label now has control characters
  and bidirectional overrides replaced.
- Table rendering sliced strings by byte (a description at 200, a URL at
  72, an earnings period), panicking when a multi-byte character such as
  "’" or "é" fell on the cut. Truncate by character.
- In JSON mode, argument errors were clap's plain text and warnings were
  plain lines ahead of the JSON error. Argument errors are now the usual
  JSON error (INVALIDINPUT, exit 2) when JSON output is selected, and
  diagnostics are JSON lines; --help and --version are unchanged.
- `config` commands ignored the output format; they follow it like every
  other command (JSON when piped or with -o json, text on a terminal).
- Ownership argument mistakes (--limit 0, --format, missing or
  conflicting import sources, --family, an empty graph root) returned
  PARSEERROR / exit 1; they are INVALIDINPUT / exit 2 as documented.
- `cache clear` followed symlinked directories and deleted files outside
  the cache; it now removes links without following them.

Output change, intentional: goldens regenerated for the JSON argument
error and the five config commands, which run with stdout piped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-17 13:24:28 +00:00
hermes
523e1409d3 fix(sync): atomic install, stored-row validation, no local artifacts from remote manifests
From the full-repo Codex audit:
- Install moved the database to a backup, then renamed the snapshot in;
  a crash between the two left no database, and the next sync deleted the
  backup first. Replace it with a single rename (old file or new, never
  neither), and restore a backup an older version left behind.
- Validation trusted the release's row_count metadata, so a snapshot with
  an empty or missing holdings table passed and replaced good data. Also
  count the rows stored for the latest release.
- A manifest fetched over HTTP(S) could name a local download_url such as
  /dev/zero, read without a size limit. Remote manifests must give an
  HTTP(S) artifact; local reads take regular files only, within the size
  limit, and manifests are capped at 1 MiB.
- A same-date build with a different row count replaced the local file
  even when it did not list it in replaces_sqlite_sha256, so a replayed
  older build could undo a corrected one. Same-date rebuilds now go only
  through replaces_sqlite_sha256, which the publisher fills.

Verified against the published snapshot: fresh install, no-op re-sync,
and forced reinstall all succeed with no leftover files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-17 12:23:58 +00:00
hermes
4357a2ba73 docs: release notes format in the release order
Notes in the style of other open-source releases: an upgrade line when
urgent, New Features / Bug Fixes / Improvements with one user-facing
sentence and PR numbers per item, and a Full Changelog compare link.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-16 10:01:47 +00:00
hermes
c6ec583edc chore: release 0.3.2
Patch release for the npm launcher fix (#41, #42): `idx` and `idx-mcp`
now install their binary on first run when the package manager skipped
postinstall, as npm 12 does under `npx` with its `allow-scripts` policy
(and pnpm 10+ and bun by default).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-16 09:54:31 +00:00
hermes
2d0bceade2 docs: news sources research: one publisher, Google News, greencloud test
- Why not a browser per user: a full engine per install and fetch, each
  user facing the challenge, IDX seeing thousands of scrapers; one
  scheduled publisher serving static files instead (diagram). Hosting
  left open (GitHub release, Cloudflare Pages/R2, a Worker).
- The IDX challenge is JavaScript-based: curl fails from a second
  datacenter (greencloud-vps, Singapore) too, so IP is not what curl
  lacks.
- Other sources measured: Google News RSS covers companies MSN misses
  (BREN 75/100 titles naming the ticker in 7 days), plain HTTP; Yahoo
  search answers 429.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-16 09:54:30 +00:00
hermes
d2ad5c142b docs: research IDX news and disclosures, propose a published feed
MSN has no company news for many tickers; IDX's disclosures cover every
issuer. idx.co.id is behind a Cloudflare challenge that plain curl and
curl-impersonate fail from the maintainer host, while the local Camofox
browser gets through. Record the working endpoints (per-issuer and
all-issuer disclosures, exchange news), their shapes and quirks, the
options, and a proposed design: a maintainer publisher writes static JSON
to a GitHub release, as the ownership snapshot does, and idx-core merges
it into news.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-16 09:54:30 +00:00
hermes
d2cd7115f1 fix(npm): no cache fallback without an absolute home directory
The release-gate Codex review found the relative-path fix incomplete:
with XDG_CACHE_HOME unset and HOME empty or relative, the fallback was
`.cache/...` under the current directory, so a checkout could plant the
binary the launcher runs. Use the home cache only when the home path is
absolute; otherwise the package's bin/ is the only location. npm smoke
plants such a binary and checks it is never run (it was, before this).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-16 09:54:30 +00:00
hermes
3041606916 fix(npm): absolute cache root, per-arch cache dir, SIGINT during install
From a second Codex review (gpt-6.1-sol, xhigh):
- A relative XDG_CACHE_HOME (invalid per the XDG spec) made the launcher
  look for, and run, a binary under the current directory, which an
  untrusted checkout could supply. Use it only when absolute.
- The cache dir now includes <platform>-<arch>, so a home directory shared
  across machines cannot hand an arm64 host an x64 binary.
- SIGINT sent to the launcher's PID alone during the first-run install is
  forwarded to the installer (it was ignored); the launcher exits 130.
- npm smoke runs the first-run install with an empty user cache and checks
  the package binary was created, so a cached binary cannot pass for it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-16 09:54:30 +00:00
hermes
907223a838 fix(npm): install to a user cache when the package is read-only; signal-safe first run
- When the package's bin/ is not writable (a root-owned `sudo npm i -g`
  whose postinstall npm 12 skipped), install the binary to a per-user
  cache ($XDG_CACHE_HOME, ~/.cache, ~/Library/Caches on macOS, under
  <package>/<version>/); the launcher looks in both. Previously the first
  run failed with EACCES and advice to rerun npm install.
- Run the first-run installer asynchronously with SIGTERM/SIGHUP
  forwarded, so a supervisor stopping the launcher stops the download;
  the installer removes its partial file and the launcher exits by the
  same signal.
- Verify an IDX_BINARY_URL local file's checksum on the copy, not the
  source, so the installed bytes are the checked ones.
- npm smoke covers the read-only fallback.

The last three come from a Codex review.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-16 09:54:30 +00:00
hermes
1049ff3155 test(agents): run opencode 2 as well as opencode 1
opencode 2 (@opencode/cli) moved MCP servers to `mcp.servers`, turns
Code Mode on by default, replaced `permission` with ordered
`permissions` rules, and dropped `run --pure` for `--standalone` (runs
otherwise go through a shared background service). Pick the config and
flag from `opencode --version`.

Its `execute` tool stays allowed: with it denied, opencode 2.0.30 never
offers the idx tools to the model in `opencode run`, though the server
connects before the first step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-16 07:40:04 +00:00
hermes
855ec0cdc3 test(agents): opencode answers only through the idx tools
opencode kept its built-in tools, so it could route around the server
under test: on published 0.3.1 it answered a quote from web search, and
on the live `changes` case it queried the ownership SQLite file with
bash and timed out. Deny every tool but `idx_*`, as Claude, pi, and
Hermes already are.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-16 06:50:29 +00:00
hermes
7ebf63ed42 test(agents): a live variant of the ownership-changes case
The `changes` case names the fixture releases (2026-01-30, 2026-02-27),
which the live snapshot does not have, so under --live/--npx it failed
agents that correctly declined to answer about missing releases and
passed ones that called the tool anyway. A case's `live` fields now
replace the mock ones; this one asks about the two most recent releases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-16 06:09:00 +00:00
hermes
ca218ef2ea test(agents): run agents through npx -y idx-mcp, as users configure it
- agent-smoke `--npx <spec>` registers `npx -y <spec>` (the published package
  or an `npm pack` tarball) with a fresh npm cache, instead of the local build,
  so each agent's own npx handling and a cold install are tested.
- `nix develop .#agents-fhs` (Linux) runs agents in an FHS sandbox: Hermes
  replaces a bare `npx` with its own generic-Linux Node, which NixOS cannot
  load. `agent-fhs <cmd>` runs anything inside it.
- Hermes: only the idx toolset (with its terminal it repaired a broken
  server mid-run and passed), the Node toolchain fetched before the first
  case (`hermes -z` waits ~15s for MCP tools), JSON config.yaml (as profiles
  write it), linked plugins for plugin model providers, and a copy of its
  mcp-stderr.log per run.
- A failed run names its cause inline, since output is deleted without --keep.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WB13omP6h7YeyJx7GNtUnF
2026-10-16 04:40:40 +00:00
hermes
bdbcf62a7b fix(npm): install the binary on first run when postinstall was skipped
npm 12 blocks install scripts of unapproved packages under `npx` and
`npm install -g` (its allow-scripts policy). The postinstall download never
ran, so `npx -y idx-mcp` exited with "installed binary not found" and MCP
clients saw only "Connection closed". Hermes Agent hits this by default: it
runs a bare `npx` with its own npm 12.

The launcher now runs the installer itself when the binary is missing,
logging to stderr so the idx-mcp stdout channel stays clean. This also covers
pnpm 10+ and bun, which skip dependency scripts by default.

npm-smoke installs idx-mcp with --ignore-scripts and checks that the first
run still starts cleanly.

Fixes #41

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WB13omP6h7YeyJx7GNtUnF
2026-10-16 04:40:40 +00:00
hermes
aac30335fc chore: release 0.3.1
Patch release for the fixes from the Codex review of 0.3.0: ticker
validation and encoded cache file names (path traversal via a ticker),
cross-process ownership DB locking, history-preserving snapshot sync,
and MCP input limits. MSRV is now 1.89.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 03:14:45 +00:00
hermes
60634d3aef fix: address final review (path traversal, sync races, MCP limits)
Findings from a Codex (gpt-6.1-sol, xhigh) review of v0.2.4..HEAD, each
verified in the code before fixing:

- high: ticker symbols reached cache file paths unchecked, so a symbol
  like `../../REPORT` read (and deleted, if unparseable) .json files
  outside the cache, reachable through MCP get_quotes. resolve_symbol now
  accepts only ticker characters, and cache keys are encoded as single
  safe path components.
- high: sync decided before downloading and replaced the database
  without rechecking; an import that landed meanwhile was erased. It now
  rechecks the local state before installing, and idx-mcp runs
  ownership_sync exclusively with ownership reads.
- replaces_sqlite_sha256 never downgrades history: the client requires
  the rebuild to keep at least the local release count, and the
  publisher lists a replaced build only then.
- cache writes are atomic (temp file + rename), so concurrent readers
  never see, and delete, a partial entry.
- idx-mcp caps symbols (20, deduplicated) and limits, and runs at most 4
  tool calls at once.
- each Yahoo client gets its own cookie jar, paired with its crumb.
- ZIP entries (XLSX, KSEI archives) are read with a 256 MiB bound;
  non-ASCII archive dates and huge percentages are parse errors instead
  of panics or overflow.
- agent-smoke --keep deletes every copied agent config, not only auth
  files.

CLI goldens unchanged; the MCP tool golden changes only to state the
new caps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 21:58:37 +00:00
hermes
5a3ce462cc fix(agents): address agent smoke review
- Require bash 4.4+ up front, and match answer facts with jq regexes
  (Oniguruma, (?i) everywhere) instead of GNU-only grep -P.
- Unparseable agent output fails that run instead of aborting the
  matrix; check_run now also reports unparseable tool calls, which
  previously passed the tool check silently.
- Copy the Hermes model block with awk up to the next top-level key, so
  a final or differently ordered block is handled.
- --keep keeps run output but deletes the copied agent credentials.
- docs: full path to the tool golden.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 21:16:06 +00:00
hermes
9d8914b2c9 test(mcp): agent smoke test across five coding agents, tool snapshot
- tests/agents/agent-smoke.sh runs tests/agents/cases.json through
  Claude Code, Codex, pi, Hermes Agent, and opencode headless against a
  local idx-mcp build, each with an isolated config, and checks the
  tools called, their key arguments, and (mock data) facts in the
  answer. Opt-in; never part of the gate.
- crates/idx-mcp/tests/golden/tools.json pins tool names, descriptions,
  and input schemas, so changes to what agents see are reviewed.
- nix develop .#agents adds Claude Code, Codex, and opencode from
  nixpkgs after the host PATH.

First run: 30/30 (5 agents x 6 cases, mock data).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 20:59:56 +00:00
hermes
58d421028f fix: address release review (WAL-aware sync hash, empty MCP inputs)
- Sync hashed only the main database file, but local edits can sit in
  the -wal sidecar, so an edited database could match an earlier build
  and be replaced. Checkpoint the WAL before hashing; if changes remain,
  never take the rebuilt-snapshot path.
- MCP tools rejected nothing for empty symbols/sections/kinds and
  returned an empty success; they now fail with INVALIDINPUT, as the CLI
  requires input.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 19:44:24 +00:00
hermes
8e3323e4f7 chore: release 0.3.0 with idx-mcp
Bump the workspace and npm packages to 0.3.0. The release workflow
builds and attaches idx-mcp binaries next to idx, the npm launcher and
installer pick their binary from package.json so npm/idx-mcp reuses
them, and the npm smoke test covers both packages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 19:01:29 +00:00
hermes
ec17a1bd05 fix(ownership): update syncs when a same-date snapshot is rebuilt
A snapshot republished for the same as-of date (the TRUE ticker fix)
kept the date, latest release, and release count, so sync saw nothing
to do and existing installs kept the bad data without --force.

The manifest gains an optional snapshot.replaces_sqlite_sha256 listing
earlier builds for that date. Queries never write to a synced file, so
a local database byte-identical to a listed build is replaced; one that
matches nothing (local imports or edits) still needs --force. The
publisher fills the list on a same-date rebuild. Older clients ignore
the field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:52:13 +00:00
hermes
cda12fe2db feat(mcp): add idx-mcp, a stdio MCP server over idx-core
Phase 4 of docs/DESIGN.md. Eleven grouped tools (quotes, history, screen,
company, analysis, compare, shareholders, investor holdings, search,
ownership changes, status); ownership_sync is listed only with
--allow-sync. Live results keep the Fetched envelope, errors use the CLI's
{error, code, message} body, and investor tools on an empty database
point at the sync path.

idx-core gains Period/Interval FromStr, Investor::status, a ticker
filter on changes, and Display for ErrorCode (now used by the CLI too).
CLI output and goldens are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:19:04 +00:00
hermes
1d8b615857 refactor(core): move ownership queries, import, and sync into idx-core
Phase 3c of docs/DESIGN.md. `ownership` handlers are now argument
parsing, one idx-core call, and rendering.

- investor.rs: Investor (also Idx::investor()) with releases, ticker,
  entity, search, cross_holders, concentration, flow, changes, graph,
  unresolved, map_alias, merge_entities, discover, sync, and import.
  Import orchestration (input resolution, download, SHA-256 dedup,
  PDF/XLSX/archive parsing, entity resolution, release write) moves
  from the CLI unchanged.
- Typed inputs: ImportSource, HolderSource, ConcentrationRank, and
  parse_report_family, with the same error messages.
- CoreConfig gains an ownership_db override; db::open_db_at opens an
  explicit path. idx-cli no longer depends on rusqlite, sha2, or ureq
  at runtime.
- Behavior note: an invalid `ticker --source` or `concentration --by`
  is now rejected before the database file is opened (and created);
  the error and exit code are unchanged.
- Fix: building idx-cli without the ownership feature no longer warns
  about the ownership-only `blue` style.

All 143 goldens unchanged; 170 core + 7 CLI unit tests and 62 CLI tests
pass; clippy clean for default, release, testing, and no-default
feature sets. Live A/B against main: 16/16 stocks identical; ownership
discover got IDX 403 on both (exit 4 vs 1 is the phase 1 change).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 09:52:17 +00:00
hermes
86cb1b4b7c refactor(core): move stocks fetching, caching, and analysis into idx-core
Phase 3b of docs/DESIGN.md. `stocks` handlers are now argument parsing,
one idx-core call, and rendering.

- Idx (client.rs) is the entry point: Idx::new(CoreConfig) with
  market(), financial(), and analysis(). Live lookups return
  Fetched<T> { data, source, fetched_at, stale } and take a CachePolicy
  (Default | Offline | Refresh, from the --offline/--no-cache flags).
- One cached_fetch replaces the CLI's three cache flows; they differed
  only in whether a fresh entry is read via get or get_offline when
  offline, which yields the same data and diagnostics. Report-building
  errors (technical) still skip the stale-cache fallback.
- market.rs: quotes, history (history_source is resolved first so the
  CLI's fallback note keeps its position), and a typed screener
  (ScreenFilter/ScreenRegion with the same error messages).
- financial.rs: profile, statements, earnings, sentiment, insights,
  news, with StatementFilter/EarningsFilter.
- analysis: TechnicalReport moves to analysis::report; Analysis offers
  technical, growth, valuation, risk, fundamental, and compare.
- CoreConfig::load() owns provider/exchange/TTL settings from
  config.toml and IDX_*; the CLI's IdxConfig adds output and color.
  With two different invalid settings at once, the reported one may
  differ from before; every single-error message is unchanged.
- Cache gains fresh/any/offline accessors returning fetch metadata.

Unit tests moved with their code (core 157 -> 168, CLI 17 -> 7). All
143 goldens unchanged; live A/B against main: 16/16 stocks identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 09:41:31 +00:00