Commit graph

16 commits

Author SHA1 Message Date
ae0ca0c109 feat: shareholder movements from the Stockbit session (ownership movements)
`idx ownership movements [SYMBOL]` and the MCP tool
`get_shareholder_movements` list reported changes in insiders' and major
holders' stakes, newest first, from Stockbit's
/insider/company/majorholder: IDX insider disclosures (any size, with the
trade price) and KSEI changes of holders at 5% and above (Stockbit's
estimated price). Stockbit filters by date range, holder id, action and
source; each page is 50 rows and one request, up to 10 pages, cached an
hour per page under every filter.

Rows keep exact signed share counts and the stake in basis points as
Stockbit rounds it; unknown action labels are kept as `other` rather than
dropped. JSON is {movements, more}; JSONL is one movement per line. The
command lives under `ownership` but needs no import, and `ownership`
commands now take --offline / --no-cache for it.

The per-symbol summary score is the next PR; docs/STOCKBIT.md records the
endpoint's parameters and action types as checked on 2026-11-03.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-03 16:02:26 +07:00
43ff4aa433 feat: corporate actions from the Stockbit session (stocks actions, stocks calendar)
`idx stocks actions SYM [--kind ...]`: one stock's cash and stock
dividends, bonus shares, rights, warrants, splits and reverse splits,
meetings, public exposes and tender offers (`--kind ipo` adds its IPO),
newest first. `idx stocks calendar [--from] [--to] [--kind ...]`: the
market's actions dated in the range (default today and 30 days ahead),
oldest first. MCP: get_corporate_actions, get_corporate_calendar.

Stockbit's `/corpaction/{type}` serves one kind per request; each type has
its own field prefix and list key, so entitlements are read by field
suffix. An action's `date` is the cum date for entitlements, else the
event's first day. Dividends keep their decimals. One request per kind,
cached 6 hours. AUTHERROR without a session.

Fixtures are the 2026-11-03 recordings (market-wide ones trimmed to
October-December). Goldens added; help-stocks and completions-bash gain
the two subcommands, MCP tools.json the two tools (regenerated with
UPDATE_GOLDEN=1). The MCP docs counted 17 tools without stockbit_login;
now 20.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-03 14:35:20 +07:00
ca4d3d09c2 feat(stockbit): one browser sign-in in idx-core for the CLI and MCP
`stockbit::signin` is now the single sign-in flow. `idx stockbit login`
waits on it in the foreground; the `stockbit_login` MCP tool runs it on a
thread and `stockbit_status` reports `sign_in.state`, so an agent knows when
to stop polling. Auto uses a browser unless the shell is over SSH or has no
display, and falls back to the local page only when the browser cannot start.

The browser way opens Stockbit's sign-in page in a profile only idx uses and
reads the session cookie over DevTools. What a real sign-in showed:

- The cookie is host-only on stockbit.com, so deleting by the assumed
  .stockbit.com removed nothing and left live tokens in the profile. It is
  now deleted by its own domain and path, with every other tab closed first,
  and read back to confirm.
- A signed-out page writes the cookie without tokens; it no longer counts.
- A profile belongs to one browser, so it records the executable that made it.
- On macOS the config directory depends on XDG_CONFIG_HOME, so a shell and an
  app started from the Dock disagreed; both default locations are now checked.
- A thread does not outlive main, so idx-mcp closes a waiting browser when
  stdin closes or on SIGTERM or Ctrl-C.

Progress goes to stderr, which keeps `-o json` to one document on stdout.

Google refused its own sign-in in this window on 2026-10-31, most likely over
the debugging flag; password sign-in works, and the docs and roadmap say so.

Claude-Session: https://claude.ai/code/session_01RqPPVaoFxLToGAXXBEoCtH
2026-10-31 23:58:21 +07:00
hermes
b5ae9b98de feat: Stockbit login for the order book, live quotes and daily history
Opt-in, with the user's own Stockbit session from a pasted
credentialStorage cookie (or IDX_STOCKBIT_REFRESH_TOKEN):

- idx stockbit login/status/logout; session file 0600, tokens never shown
- rotating refresh under a cross-process lock; one refresh+retry on 401
- stocks orderbook: depth with lots and orders, OHLC, value, foreign flow
- --provider stockbit quotes from the session API when logged in
- --history-provider stockbit, and auto under the stockbit provider for
  5d-1y daily windows
- MCP get_orderbook and stockbit_status (17 tools)

Endpoints, units and recorded responses from fzn0x/idx (MIT).
2026-10-23 08:35:57 +00:00
hermes
74e5bb0536 feat: login-free Stockbit intraday and quote provider
Read Stockbit's public symbol page (__NEXT_DATA__) with a plain ureq
request and an honest user agent: `stocks intraday` (today by the
minute, any provider), `--provider stockbit` quotes with the best
bid/offer only while the market is open, and MCP `get_intraday`.

- UPSTREAMBLOCKED error (exit 4) for a CDN 403; never retried
- requests paced 1 s apart across idx processes, 5 s snapshot cache
- global --provider flag (yahoo, msn, stockbit)
- roadmap: L1 ships from Stockbit; depth stays a login follow-on
2026-10-23 04:58:32 +00:00
hermes
8b507d78e2 feat: foreign flow from IPOT by default (--source ipot|idx)
`idx stocks flow` now reads IPOT by default, which needs nothing installed:
per session, the all-investor broker summary gives the value traded and
the foreign one gives foreign buying (its footer), the foreign net and
selling (buying less the net), across all boards, through the broker
summary cache. IPOT's foreign net matched IDX's estimate within 0.3 B on
BBCA 2026-10-12 to 16.

`--source idx` (and `source` on get_foreign_flow) keeps IDX's trading
summary: exact share counts, close and the non-regular share, behind
curl-impersonate and Cloudflare.

- ForeignFlowDay gains foreign_buy_value, foreign_sell_value and
  foreign_share_bps for both sources (estimated at the session average
  for IDX); ForeignFlow gains source, the value totals and rounded.
- The table lays out each source's figures; the live smoke runs both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 06:40:22 +00:00
hermes
29f143fa1b feat: foreign flow (idx stocks flow, get_foreign_flow)
`idx stocks flow SYMBOL` shows what foreign investors bought and sold of a
stock, session by session, from IDX's own daily trading summary: foreign
buy, sell and net volume, net value at the session's average price, and
the share of value traded outside the regular board, with range totals.
Latest 10 sessions by default (--days, at most 60) or --from/--to. The MCP
server exposes it as get_foreign_flow.

- api/idx: pure parser for GetStockSummary (rows dated another session
  are dropped) and a provider over curl-impersonate's profile fallback
  with a 30 s limit; Cloudflare passes only some fingerprints.
- flow: each session is cached whole by date, so other stocks' flows for
  the same sessions need no request. Weekends are never requested.
- Idx::idx_code is shared by broker and flow for the .JK-only check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 05:39:12 +00:00
hermes
970da26a0e feat: bandarmology (idx stocks bandar, analyze_bandar)
`idx stocks bandar SYMBOL...` reads the broker summary: for the top 1, 3,
5 and 10 brokers per side, buy value minus sell value in basis points of
value traded, with a signal from big accumulation to big distribution on
the headline tier (--top, default 5). Several symbols give a screen ranked
by that figure (at most 100, one request each), which pipes from
`stocks screen`. The MCP server exposes it as analyze_bandar (20 symbols).

- analysis/bandar.rs is pure and versioned (`top_concentration_v1`):
  integer basis points, no figure missing counted as zero, buy/sell VWAP
  from exact averages, and the brokers on both sides listed because their
  gross figures overstate their direction.
- Thresholds (±5%/±15%) follow fzn0x/idx (MIT); docs call them heuristics.
- Invalid input fails a screen at once; other errors fail one symbol and
  are warned about on stderr.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 05:34:37 +00:00
hermes
860d4daf77 feat(cli,mcp): idx stocks broker and get_broker_summary
`idx stocks broker SYMBOL` shows the top 10 buyers and sellers of a stock
by lots for the latest session with trades, one `--date`, or a
`--from`/`--to` range, filtered by `--investor` and `--board`. The MCP
server exposes the same as get_broker_summary.

- Range and filter parsing lives in idx-core (BrokerRange::from_args,
  FromStr for the filters), so both frontends accept the same input.
- Without dates, the latest session is found by walking back over
  weekdays; offline, uncached days are skipped.
- Table output lists foreign and state-owned brokers, and says when the
  source rounded figures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-18 05:33:12 +00:00
hermes
1d3fe5ca88 fix(ownership): correct changes, alias search, graph, and name lookups
From a full-repo Codex audit, each reproduced on the synced ownership DB:
- `changes` paired an investor's accounts across releases instead of
  comparing totals: KREN's KRESNA PRIMA INVEST went 9.44% -> 13.45% (a
  second 4.01% account) but was reported as decreased 944 -> 401 bps.
  Sum a holder's accounts per ticker, and read one release per date (the
  latest imported, as other queries do), so a corrected file supersedes
  the original instead of mixing with it.
- `changes` / get_ownership_changes accepted any dates: `from: ""`
  reported all 7,154 holders as new, unknown dates returned no changes.
  Dates must be YYYY-MM-DD imported releases with `from` before `to`.
- get_investor_holdings / `entity` with an empty name matched every
  investor and returned an arbitrary one ("AMY"); `%` and `_` acted as
  LIKE wildcards. Reject empty names; match wildcards literally.
- Alias search never worked: the FTS table declared an `aliases` column
  its content table lacks (`no such column: T.aliases`), so searches
  fell back to canonical names and a raw KSEI name such as
  "PT ASTRA INTERNATIONAL TBK" found nothing. Search names and aliases
  directly, ranked exact name, exact alias, prefix, then shorter names.
- `graph` enumerated every path (exponential in depth) and its cycle
  check matched substrings, dropping entity:1 when entity:10 was on the
  path. Traverse breadth-first with an exact visited set.
- Name normalization removed leading PT tokens from the front of a Vec,
  quadratic in hostile names; narrow a slice instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-17 04:09:51 +00:00
hermes
01f181689e test(mcp): pin paging past the end and the per-page summary
From the Pullfrog review of #48: an offset at or past `total` returns an
empty page with `next_offset: null`, and every page carries the same
`summary` (it covers all rows). Assert both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-16 09:54:30 +00:00
hermes
db7c90861c feat(mcp): page get_ownership_changes and summarize all changes
Between the two latest releases there are 1,568 holder changes. With only
`limit` (up to 2000, ~200 KB), agents that wanted them all overflowed
their tool output: Hermes then paged ticker by ticker (32 calls), and
opencode 1 and 2 did the same and timed out.

- `offset` pages through the rows; responses carry `offset` and
  `next_offset` (null after the last page). Pages cap at 300 rows
  (~40 KB); the over-limit error points to `offset` and `summary`.
- Rows come largest |delta| first, so the first page holds the moves
  that matter rather than tickers starting with A.
- `summary` covers every row: counts by change type, the number of
  tickers, and the ten tickers with the most changes.

The CLI's `ownership changes` is unchanged. tools.json and the website's
MCP pages are regenerated for the new description and parameter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-16 09:54:30 +00:00
hermes
60634d3aef fix: address final review (path traversal, sync races, MCP limits)
Findings from a Codex (gpt-6.1-sol, xhigh) review of v0.2.4..HEAD, each
verified in the code before fixing:

- high: ticker symbols reached cache file paths unchecked, so a symbol
  like `../../REPORT` read (and deleted, if unparseable) .json files
  outside the cache, reachable through MCP get_quotes. resolve_symbol now
  accepts only ticker characters, and cache keys are encoded as single
  safe path components.
- high: sync decided before downloading and replaced the database
  without rechecking; an import that landed meanwhile was erased. It now
  rechecks the local state before installing, and idx-mcp runs
  ownership_sync exclusively with ownership reads.
- replaces_sqlite_sha256 never downgrades history: the client requires
  the rebuild to keep at least the local release count, and the
  publisher lists a replaced build only then.
- cache writes are atomic (temp file + rename), so concurrent readers
  never see, and delete, a partial entry.
- idx-mcp caps symbols (20, deduplicated) and limits, and runs at most 4
  tool calls at once.
- each Yahoo client gets its own cookie jar, paired with its crumb.
- ZIP entries (XLSX, KSEI archives) are read with a 256 MiB bound;
  non-ASCII archive dates and huge percentages are parse errors instead
  of panics or overflow.
- agent-smoke --keep deletes every copied agent config, not only auth
  files.

CLI goldens unchanged; the MCP tool golden changes only to state the
new caps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 21:58:37 +00:00
hermes
9d8914b2c9 test(mcp): agent smoke test across five coding agents, tool snapshot
- tests/agents/agent-smoke.sh runs tests/agents/cases.json through
  Claude Code, Codex, pi, Hermes Agent, and opencode headless against a
  local idx-mcp build, each with an isolated config, and checks the
  tools called, their key arguments, and (mock data) facts in the
  answer. Opt-in; never part of the gate.
- crates/idx-mcp/tests/golden/tools.json pins tool names, descriptions,
  and input schemas, so changes to what agents see are reviewed.
- nix develop .#agents adds Claude Code, Codex, and opencode from
  nixpkgs after the host PATH.

First run: 30/30 (5 agents x 6 cases, mock data).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 20:59:56 +00:00
hermes
58d421028f fix: address release review (WAL-aware sync hash, empty MCP inputs)
- Sync hashed only the main database file, but local edits can sit in
  the -wal sidecar, so an edited database could match an earlier build
  and be replaced. Checkpoint the WAL before hashing; if changes remain,
  never take the rebuilt-snapshot path.
- MCP tools rejected nothing for empty symbols/sections/kinds and
  returned an empty success; they now fail with INVALIDINPUT, as the CLI
  requires input.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 19:44:24 +00:00
hermes
cda12fe2db feat(mcp): add idx-mcp, a stdio MCP server over idx-core
Phase 4 of docs/DESIGN.md. Eleven grouped tools (quotes, history, screen,
company, analysis, compare, shareholders, investor holdings, search,
ownership changes, status); ownership_sync is listed only with
--allow-sync. Live results keep the Fetched envelope, errors use the CLI's
{error, code, message} body, and investor tools on an empty database
point at the sync path.

idx-core gains Period/Interval FromStr, Investor::status, a ticker
filter on changes, and Display for ErrorCode (now used by the CLI too).
CLI output and goldens are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:19:04 +00:00