Commit graph

324 commits

Author SHA1 Message Date
dogfiles
f74c271c1f
Merge pull request #78 from 0xrsydn/feat/stockbit-corpaction
feat: corporate actions from the Stockbit session (stocks actions, stocks calendar)
2026-11-05 17:53:09 +07:00
dogfiles
48b2206db4
Merge pull request #80 from 0xrsydn/feat/stockbit-movements
feat: shareholder movements from the Stockbit session (ownership movements)
2026-11-05 17:53:00 +07:00
hermes
32b978b16c docs(movements): say the JSON can carry incomplete 2026-11-05 10:22:04 +00:00
dogfiles
c2336ea52f
Merge pull request #81 from 0xrsydn/fix/mcp-warnings
fix(mcp): hand a call's warnings to the agent
2026-11-05 17:21:47 +07:00
0aed926f4c Merge branch 'feat/stockbit-movements' into fix/mcp-warnings
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 21:05:07 +07:00
9373be222a test(movements): check the allowlist against the real movements path builder
The read-path test listed a hand-written movements path; it now also builds
paths with movement::request_path, so the test fails offline if the
allowlist and the builder drift apart (Pullfrog on #80).

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 21:05:01 +07:00
96a900e0b8 fix(calendar): say when the table holds offers opened before --from
Open offers are listed by the day they opened, so "from 2026-11-03 to
2026-11-12" sat above tenders dated in October (Pullfrog on #78). The title
adds "with offers opened earlier and still open" when such rows are there.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 21:04:24 +07:00
edfcf0bef9 Merge branch 'feat/stockbit-corpaction' into feat/stockbit-movements
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 21:04:24 +07:00
7830de3636 docs(mcp): the generated tools page names warnings
The tools reference page described results as {data, source, fetched_at,
stale} and failures as {error, code, message}, without the `warnings` this
PR adds (Pullfrog on #81). The generator's intro says so; page regenerated.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 20:46:27 +07:00
0582585004 Merge branch 'feat/stockbit-movements' into fix/mcp-warnings
Kept both helpers where exodus.rs conflicted: blocked() from this branch and is_read_path() from #78.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 20:45:20 +07:00
8f5c56e464 fix(movements): allow the movements path, and describe its one cache entry
- The session client's new read-path allowlist (#78) gains
  `/insider/company/majorholder?`, the one endpoint movements reads; without
  it every live movements query would fail before leaving the process. The
  mock does not go through the client, so its tests could not show this.
- docs/ARCHITECTURE.md described one cache entry per page: a query's pages
  are read and cached together, keyed by the page count, and a failed later
  page returns the rows before it with `incomplete` (Pullfrog on #80).

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 20:44:51 +07:00
75da3ff235 Merge branch 'feat/stockbit-corpaction' into feat/stockbit-movements
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 20:44:15 +07:00
884c6459e4 fix(actions): keep small foreign dividends, allow only read paths, show open offers
From Pullfrog's review of #78:

- A dividend in another currency keeps 6 decimals (USD 0.0012 was rounded
  to 0.0 in JSON and "USD 0" in the table); rupiah stays to the sen.
- `--kind stock_dividend` and `--kind reverse_split` work on the CLI too,
  as they already did over MCP.
- The session client GETs only the prefixes in READ_PATHS, as
  docs/STOCKBIT.md's rule 1 already said: any other path (orders, PIN,
  withdrawals) fails before a request or token refresh leaves the process.
- `stocks calendar` and get_corporate_calendar also list rights trading,
  tender offers and IPO offers still open in the range, so an offer that
  began earlier shows while it can be acted on. A warrant's years-long
  windows do not count. The 2026-11-03..12 golden gains DOOH, NAYZ and AUTO.
- `stocks calendar --help` names the ex-date fallback, and docs/STOCKBIT.md
  shows the record and surface as shipped.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 20:44:07 +07:00
4b56f8c516 Merge branch 'feat/stockbit-login' into feat/stockbit-corpaction
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 20:38:04 +07:00
9590219977 fix(stockbit): end a browser sign-in on a CDN block
A block (UPSTREAMBLOCKED) was treated as a passing failure, so idx's browser
re-checked the same cookie every 5 s until the 15-minute timeout, warning
"does not retry a block ... retrying". A block holds for every session from
this network: the wait now ends with it at once.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 20:37:54 +07:00
9b46161566 fix(mcp): hand a call's warnings to the agent
idx-core reports what it works around (history falling back to yahoo when
Stockbit refuses the session, a stale cache served after a failed fetch, a
movements query cut short) as warnings on stderr. An agent never reads
idx-mcp's stderr, so it saw `source: "yahoo"` or `stale: true` with no reason.

- Each tool call collects its warnings with runtime::on_warning and returns
  them as `warnings` on the answer or the error, when there are any.
- The stale-cache warning names the failure behind it.
- The mock's session API 403 is the real session API's block message; the
  public page's block names IDX_PROVIDER=msn as well as `--provider msn`.

Goldens: error-intraday-blocked (json, table) carry the reworded block hint.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 19:26:10 +07:00
e739ee3c20 Merge branch 'feat/stockbit-corpaction' into feat/stockbit-movements
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 19:23:53 +07:00
a0bf478ec2 fix(actions): take the old kind names as input too
JSON `kind` now uses the `--kind` names, so an agent that copies a kind from
an answer cached or read before that change (`public_expose`,
`tender_offer`) would get INVALIDINPUT. Both are accepted, as hidden `--kind`
aliases and by the MCP tools.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 19:23:48 +07:00
4dee7fe4dd Merge branch 'feat/stockbit-login' into feat/stockbit-corpaction
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 19:22:25 +07:00
362d314118 fix(stockbit): tell a polling agent when Stockbit refuses the sign-in
A refused cookie in idx's browser was only a stderr warning, which nobody
reads under idx-mcp: stockbit_status showed `waiting` until the timeout, and
the timeout message did not mention the refusal.

- runtime::on_warning sends a thread's warnings to a sink as well as
  stderr, until its guard drops.
- The background sign-in puts its warnings in `sign_in.warning`, cleared
  once it signs in.
- The errors that end a browser wait (timeout, window closed, browser
  exited) name a refusal it hit.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 19:22:14 +07:00
0c7320a343 fix(movements): read a query's pages as one answer and keep a partial one
- Each page was cached on its own, so a multi-page answer could join
  pages read at different times from a feed that gains rows at the top,
  repeating some rows and missing others. A query's pages are now read
  together and cached as one entry.
- A failed later page threw away the pages already read. They are now
  returned with incomplete saying why, and not cached.
- The library did not cap pages at MAX_PAGES itself; it clamps now.
- The hint said to raise --pages even at 10; at 10 it says to narrow the
  dates. -o jsonl, which streams rows only, gives the hint on stderr.
- A duplicate match arm in the CLI was dead; it is gone.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 17:39:10 +07:00
5a6b5d1e2e Merge branch 'feat/stockbit-corpaction' into feat/stockbit-movements
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 17:36:04 +07:00
1e5561e990 fix(actions): JSON kinds match --kind, and harden parsing and ranges
- JSON named kinds by their Rust names (public_expose, tender_offer,
  stock_dividend, reverse_split) while --kind and MCP take expose, tender,
  stock-dividend, reverse-split, so an output kind could not be passed
  back. JSON now uses the input names; old cache entries still read, and
  --kind also accepts underscores.
- {"dividend": null, "today": "..."} failed to parse and aborted the
  whole command; a null list beside scalars now reads as none.
- An extreme --from panicked on the default 30 days; it is an error now.
  The 366-day limit let 367 days through; it counts both ends now.
- An entitlement without a cum date sat on its ex date under a note that
  calls DATE the cum date. The table marks it (ex) and says what that
  means; JSON leaves cum_date out, as before, and the docs say so.
- A dividend's currency was ignored; one not in IDR now carries currency
  and the table shows it instead of Rp.
- The live check no longer fails a quiet month's empty calendar.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 17:35:51 +07:00
6ad57dd997 Merge branch 'feat/stockbit-login' into feat/stockbit-corpaction
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 17:31:35 +07:00
c03c0f41f7 fix(quote): market_status reads open or close with a login too
Logged-in quotes passed Stockbit's session state through verbatim
(STATE_NAME_SESSION_2), while public-page quotes say open or close, so a
consumer checking for "open" broke after login. Quotes now use the two
words either way; the order book keeps the state verbatim.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 17:31:20 +07:00
63587e4dc2 fix(history): fall back to Yahoo when Stockbit refuses the session
auto history picks Stockbit when a session file exists, so an expired
session failed history and technical with AUTHERROR. auto now retries on
Yahoo with a warning; an explicit --history-provider stockbit still fails.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 17:31:20 +07:00
e7f1fa2c90 fix(stockbit): keep a working session through a failed login check
- login restored the previous session on any check error, even after the
  check had rotated the pasted session: a timeout then left no working
  session. It now keeps a rotated session unless Stockbit refused it.
- The browser sign-in marked a cookie refused on any error, so a 429 or a
  timeout held a signed-in user until the wait ran out. Passing errors
  are retried after a pause; only a refusal is final.
- The local page checked concurrent pastes on their own threads, so a
  later one could overwrite the session the first had just rotated.
  Pastes are now checked one at a time, and none after one is accepted.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 17:31:20 +07:00
81a3fe7266 Merge branch 'feat/stockbit' into feat/stockbit-login
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 17:24:55 +07:00
6f8d3c071a fix(intraday): mark a chart from before the latest session
stocks intraday never set quote.behind_session, so on a holiday or when
Stockbit lags, a past session's chart read as current. Mark it as quotes
do and print the same note under the header.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-04 17:24:42 +07:00
615083ee0b test(smoke): live case for ownership movements with a Stockbit session
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-03 16:02:26 +07:00
ae0ca0c109 feat: shareholder movements from the Stockbit session (ownership movements)
`idx ownership movements [SYMBOL]` and the MCP tool
`get_shareholder_movements` list reported changes in insiders' and major
holders' stakes, newest first, from Stockbit's
/insider/company/majorholder: IDX insider disclosures (any size, with the
trade price) and KSEI changes of holders at 5% and above (Stockbit's
estimated price). Stockbit filters by date range, holder id, action and
source; each page is 50 rows and one request, up to 10 pages, cached an
hour per page under every filter.

Rows keep exact signed share counts and the stake in basis points as
Stockbit rounds it; unknown action labels are kept as `other` rather than
dropped. JSON is {movements, more}; JSONL is one movement per line. The
command lives under `ownership` but needs no import, and `ownership`
commands now take --offline / --no-cache for it.

The per-symbol summary score is the next PR; docs/STOCKBIT.md records the
endpoint's parameters and action types as checked on 2026-11-03.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-03 16:02:26 +07:00
1b72c3ed6b docs(smoke): the Stockbit session cases and why they share the real config
Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-03 14:38:04 +07:00
73735767bb test(smoke): live cases for the Stockbit session (orderbook, actions, calendar)
Run only when `idx stockbit status` reports a session. They keep the
user's real config directory, which holds the session: an isolated copy
would refresh on its own and spend the user's rotating refresh token.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-03 14:37:38 +07:00
43ff4aa433 feat: corporate actions from the Stockbit session (stocks actions, stocks calendar)
`idx stocks actions SYM [--kind ...]`: one stock's cash and stock
dividends, bonus shares, rights, warrants, splits and reverse splits,
meetings, public exposes and tender offers (`--kind ipo` adds its IPO),
newest first. `idx stocks calendar [--from] [--to] [--kind ...]`: the
market's actions dated in the range (default today and 30 days ahead),
oldest first. MCP: get_corporate_actions, get_corporate_calendar.

Stockbit's `/corpaction/{type}` serves one kind per request; each type has
its own field prefix and list key, so entitlements are read by field
suffix. An action's `date` is the cum date for entitlements, else the
event's first day. Dividends keep their decimals. One request per kind,
cached 6 hours. AUTHERROR without a session.

Fixtures are the 2026-11-03 recordings (market-wide ones trimmed to
October-December). Goldens added; help-stocks and completions-bash gain
the two subcommands, MCP tools.json the two tools (regenerated with
UPDATE_GOLDEN=1). The MCP docs counted 17 tools without stockbit_login;
now 20.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-03 14:35:20 +07:00
8a1b29043c docs: Stockbit session endpoints and the design of features on them
docs/STOCKBIT.md: every market-data endpoint in Stockbit's web client,
probed with a session on 2026-11-03 (corporate actions, shareholder
movements, composition, market detector, analyst consensus, hourly charts,
market states), the read-only rules all session features follow, and the
design of corporate actions and shareholder movements. Linked from
AGENTS.md and ROADMAP.md.

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-03 14:04:32 +07:00
32f476bae7 fix(stockbit): bid/offer while trading; drop stale foreign flow
Observed in session on 2026-11-03:

- The regular board trades as STATE_NAME_SESSION_2, not the assumed
  STATE_NAME_MARKET_OPEN, so session quotes hid bid/offer all day. Any
  numbered STATE_NAME_SESSION_<n> now counts as trading; the lunch break
  is STATE_NAME_BREAK.
- In session the order book's fbuy/fsell/fnet were the previous session's
  (equal to IPOT's 2026-11-02 net for BBCA, BBRI and TLKM) and
  has_foreign_bs was false. Foreign figures are kept only when that flag
  is true.

Fixtures for the break, the trading state and an in-session book; the
table names the session ("market open (session 2)").

Claude-Session: https://claude.ai/code/session_01RqPPVeVDrnLoP7NUZmLRxY
2026-11-03 14:02:03 +07:00
ca4d3d09c2 feat(stockbit): one browser sign-in in idx-core for the CLI and MCP
`stockbit::signin` is now the single sign-in flow. `idx stockbit login`
waits on it in the foreground; the `stockbit_login` MCP tool runs it on a
thread and `stockbit_status` reports `sign_in.state`, so an agent knows when
to stop polling. Auto uses a browser unless the shell is over SSH or has no
display, and falls back to the local page only when the browser cannot start.

The browser way opens Stockbit's sign-in page in a profile only idx uses and
reads the session cookie over DevTools. What a real sign-in showed:

- The cookie is host-only on stockbit.com, so deleting by the assumed
  .stockbit.com removed nothing and left live tokens in the profile. It is
  now deleted by its own domain and path, with every other tab closed first,
  and read back to confirm.
- A signed-out page writes the cookie without tokens; it no longer counts.
- A profile belongs to one browser, so it records the executable that made it.
- On macOS the config directory depends on XDG_CONFIG_HOME, so a shell and an
  app started from the Dock disagreed; both default locations are now checked.
- A thread does not outlive main, so idx-mcp closes a waiting browser when
  stdin closes or on SIGTERM or Ctrl-C.

Progress goes to stderr, which keeps `-o json` to one document on stdout.

Google refused its own sign-in in this window on 2026-10-31, most likely over
the debugging flag; password sign-in works, and the docs and roadmap say so.

Claude-Session: https://claude.ai/code/session_01RqPPVaoFxLToGAXXBEoCtH
2026-10-31 23:58:21 +07:00
hermes
cb532b670b feat(stockbit): print the ssh forward when the shell is remote
The login page listens on 127.0.0.1, so a browser on the Mac driving a VM
could never reach it. Over SSH, idx now picks port 7878, skips opening a
browser on the VM, and prints the exact ssh -L to run where the browser
is.
2026-10-23 13:52:15 +00:00
hermes
00c59ad9c9 fix(stockbit): answer login-page connections concurrently
Browsers open idle preconnects; serving one connection at a time made the
page wait out the 3 s read timeout on each, which read as "can't
connect". Each connection now gets its own thread (capped) and a silent
one is dropped. Regression test; measured 10.1 s -> 0.08 s with three
idle sockets. Docs cover a browser on another machine (--port + ssh -L).
2026-10-23 13:28:12 +00:00
hermes
12fdc8c8e8 feat(stockbit): connect a session from a local browser page
idx stockbit login in a terminal now serves a page on 127.0.0.1: a
bookmarklet that reads stockbit.com's session cookie in the user's own
browser and opens the page with it in the URL fragment, or a
read-the-clipboard button, with the manual paste as the fallback. A
per-run secret guards the path, the page is no-store with a strict CSP,
and it stops on success or after five minutes. --no-browser prints the
URL (--port pins it for ssh -L); piped stdin keeps the paste path.

A paste Stockbit refuses now restores the previous session instead of
losing it.
2026-10-23 12:29:01 +00:00
hermes
1d4ab65853 docs: Stockbit order-book sizes are shares, checked against a live session 2026-10-23 10:09:43 +00:00
hermes
b5ae9b98de feat: Stockbit login for the order book, live quotes and daily history
Opt-in, with the user's own Stockbit session from a pasted
credentialStorage cookie (or IDX_STOCKBIT_REFRESH_TOKEN):

- idx stockbit login/status/logout; session file 0600, tokens never shown
- rotating refresh under a cross-process lock; one refresh+retry on 401
- stocks orderbook: depth with lots and orders, OHLC, value, foreign flow
- --provider stockbit quotes from the session API when logged in
- --history-provider stockbit, and auto under the stockbit provider for
  5d-1y daily windows
- MCP get_orderbook and stockbit_status (17 tools)

Endpoints, units and recorded responses from fzn0x/idx (MIT).
2026-10-23 08:35:57 +00:00
hermes
570439369e docs(web): Stockbit intraday on the quickstart, landing page and agent notes 2026-10-23 07:15:24 +00:00
hermes
74e5bb0536 feat: login-free Stockbit intraday and quote provider
Read Stockbit's public symbol page (__NEXT_DATA__) with a plain ureq
request and an honest user agent: `stocks intraday` (today by the
minute, any provider), `--provider stockbit` quotes with the best
bid/offer only while the market is open, and MCP `get_intraday`.

- UPSTREAMBLOCKED error (exit 4) for a CDN 403; never retried
- requests paced 1 s apart across idx processes, 5 s snapshot cache
- global --provider flag (yahoo, msn, stockbit)
- roadmap: L1 ships from Stockbit; depth stays a login follow-on
2026-10-23 04:58:32 +00:00
018d9ba936 docs: catch up the site and README with v0.4.0
The MCP overview and README still counted 11 tools and left out broker
summaries, bandarmology and foreign flow; the providers note still said
foreign flow comes from IDX, though IPOT is the default since #68; the
quickstart had no example of the new commands; and the stocks guide never
explained the behind-session note on quotes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012YsmjNoLBrGi9daeU3kj7k
2026-10-20 16:10:46 +07:00
brutaljokerz
a082a49f97
Merge pull request #72 from 0xrsydn/release/v0.4.0 v0.4.0
release: 0.4.0
2026-10-19 18:00:44 +07:00
hermes
160014c504 chore: release 0.4.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-19 10:50:06 +00:00
brutaljokerz
64c25ad0a4
Merge pull request #71 from 0xrsydn/feat/quote-behind-session
feat: flag quotes behind the latest IDX session (#50)
2026-10-19 17:43:38 +07:00
hermes
054dd9df54 test(cli): a non-IDX quote is never behind the IDX session
From Pullfrog's review of 834e47b: nothing exercised the `.JK` guard on
quotes. The cache-entry test now also reads AAPL.US with an old as_of
and expects behind_session false.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-19 10:37:57 +00:00
hermes
834e47b336 fix: mark only IDX quotes behind the session; test the wiring
Review findings on #71:
- The IDX session rule flagged non-IDX data: US screen rows on Monday
  evening WIB, and JP/HK/SG rows on their own holidays. Only `.JK`
  quotes and `--region id` screens are marked now.
- Nothing tested the marking call sites. A screen test checks IDX rows
  online and offline, and US rows unmarked. A quote test reads a cache
  entry in 0.3.3's format (no field) offline.
- The get_quotes description now says the flag also means the stock did
  not trade, or an exchange holiday; screen_stocks mentions the field.
- latest_session is pub(crate).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3syn2fN2LNo5pvTSqc6eh
2026-10-19 10:26:09 +00:00