0xrsydn/sts2-bot
1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add design doc and research notes

Browse source 
DESIGN.md covers the three-layer architecture (facts in code, Jev for
tactics, gated escalation for macro). research/ documents the engine and
mod surface, the Jev classifier's measured behavior, the STS2MCP HTTP
interface, state shapes, failure modes, decision architecture, and a
run log of the first four sessions.
This commit is contained in:
0xrsydn 2026-09-21 17:09:43 +07:00
commit fb32822468
9 changed files with 2341 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

374
docs/DESIGN.md Normal file
View file

@ -0,0 +1,374 @@
# STS2 Bot — Design (verified 2026)
Jev (TypeSafe System One) as the tactics policy. Confidence-gated LLM escalation for macro.
## Verified environment
| Item | Value |
|---|---|
| Game build | v0.107.1, commit 59260271 |
| Engine | Godot 4.5.1 (.NET), .NET 9.0.7, osx-arm64 |
| Main assembly | `sts2.dll` + `sts2.xml` (19,635 documented members) |
| Mods dir | `SlayTheSpire2.app/Contents/MacOS/mods/` (absent) |
| Workshop dir | `Steam/steamapps/workshop/content/2868840/` (absent) |
| TypeSafe API | live; needs `TYPESAFE_API_KEY` (unset) |
| STS2MCP | v0.4.0, tested against game v0.103.2 |
| Python SDKs | none installed; Node available |
## Manifest schema (JSON, from STS2_MCP.json)
{"id","name","author","description","version","has_pck","has_dll","affects_gameplay"}
Note: snake_case in JSON, not the C# property names.
## Actuator: STS2MCP
Base URL `http://localhost:15526`. No auth.
- `GET /api/v1/singleplayer?format=json|markdown` — read state
- `POST /api/v1/singleplayer` — `{"action": ..., ...}`
- `GET /api/v1/compendium`, `/profile`, `/wiki?query=`, `/profiles`
Response always has `state_type`, `run{act,floor,ascension}`, `player`.
### state_type -> legal actions (this IS the action enumerator)
menu, game_over -> menu_select
monster/elite/boss -> play_card, use_potion, end_turn
hand_select -> combat_select_card, combat_confirm_selection
rewards -> claim_reward, proceed
card_reward -> select_card_reward, skip_card_reward
map -> choose_map_node
event -> choose_event_option, advance_dialogue
rest_site -> choose_rest_option, proceed
shop/fake_merchant -> shop_purchase, proceed
treasure -> claim_treasure_relic, proceed
card_select -> select_card, confirm_selection, cancel_selection
bundle_select -> select_bundle, confirm_bundle_selection, cancel_bundle_selection
relic_select -> select_relic, skip_relic_selection
crystal_sphere -> crystal_sphere_set_tool/click_cell/proceed
overlay, unknown -> none (manual)
`use_potion` / `discard_potion` work in any state where potions are reachable.
Card rules text ships in game state when cards are visible.
## Decision loop
Card indices shift on every play. The loop is strictly closed:
observe -> compute facts (pure code) -> ask Jev (1 batched call)
-> gate on confidence -> execute ONE action -> re-observe
Never precompute an action list.
## Three layers
| Layer | Engine | Responsibility |
|---|---|---|
| Facts | pure Python | arithmetic, legality, deck stats, threat model |
| Tactics | Jev | per-turn card play, targeting, potions, event options |
| Macro | scripted -> LLM | archetype, card rewards, routing, boss prep |
| Gate | confidence threshold | when to escalate to Claude/GPT |
Critical: Jev cannot count or do arithmetic (documented jaggedness).
Compute numbers in code. Send conclusions, not raw values.
## Jev primitives
- Choice: pick 1 of <=255 options -> choice, probabilities, confidence
- Score: position on ordered levels -> score, legend, probabilities, confidence
- Noul: yes/no -> noul (probability only, no confidence)
Batch all questions for one state into ONE call. Parallel, ~100ms,
near-free latency, output tokens free. 12.2x cheaper than separate calls.
Limits: 64k context total; 32k state + longest question. Text only.
Price $0.042/Mtok input. Rate 250k tok/s, 1200 req/min.
## Combat question template (one batched call)
lethal_this_turn Noul any line in hand kills all enemies
must_block Noul preventing damage beats dealing damage
best_play Choice one option per legal play, described in words
target Choice which enemy to hit
Precedence in code: lethal -> must_block -> best_play.
Confidence < threshold -> fall back to AGENTS.md heuristics.
## Cost projection
| Metric | Estimate |
|---|---|
| Tokens/run | ~2M filtered |
| Cost/run | ~$0.08-0.35 |
| Model time/run | ~100-150 s |
| Wall clock/run | ~5-15 min |
| Frontier LLM agent (reported) | ~8M tok, ~$20-40/run |
## Risks
1. Version drift: STS2MCP tested on v0.103.2, game is v0.107.1.
2. Jev arithmetic/counting failure -> mitigate by computing in code.
3. Context rot -> send filtered state, never a raw dump.
4. Index shift -> strictly closed loop, re-read after each action.
5. Macro plateau -> confidence-gated escalation to a reasoning LLM.
6. Doc drift: repo AGENTS.md uses MCP tool names (`combat_end_turn`,
`rewards_pick_card`); HTTP uses (`end_turn`, `select_card_reward`).
Trust raw-simplified.md for direct HTTP use.
## MEASURED RESULTS (validated against live API)
### Credential
Key found at `~/.config/secrets/global-env/TYPESAFEAI_API_KEY`
(sops-nix symlink -> `~/.config/sops-nix/secrets/`). NOT in the shell
environment. Read from that path at runtime; never log the value.
### Validation
First call: HTTP 200, `model: jev-1.13.0`. Game-correct answers on a
combat scenario (Cleave into the 12 HP attacker, confidence 0.85).
### Latency (measured, NOT the documented 100 ms)
| Request | Time |
|---|---|
| Minimal (1 short noul) | 0.73 s |
| Minimal, 5 repeats | 0.73 / 0.73 / 0.74 / 0.80 / 0.76 s |
| Full combat state, 3 questions | 0.90 s |
| GET /v1/models ttfb | 0.68 s (connect 0.22 s) |
Floor is ~0.73 s, dominated by network + server TTFB.
State size and question count are nearly free -> fan-out is confirmed.
Revised budget: ~0.75 s per action call. ~500-600 actions per run
-> ~7-8 min pure API time. Wall clock per run: ~20-45 min.
### Arithmetic failure CONFIRMED (critical)
Ground truth: energy 3, hand Strike(1c/6dmg) x3 + Bash(2c/8dmg),
target 19 HP. Max reachable damage = 18. Lethal = NO.
max_damage_bucket -> "18_to_23" (0.73) CORRECT
lethal_available -> 0.79 WRONG (true answer: no)
The model bucketed the magnitude correctly but failed the threshold
comparison (18 vs 19).
CRITICAL IMPLICATION: confidence was 0.79 on a wrong answer.
Confidence gating does NOT protect against arithmetic errors.
All arithmetic (lethal, block deficit, damage totals) MUST be computed
in code and passed in as conclusions. Never ask Jev to compare numbers.
### Bundle signing (macOS)
App is adhoc-signed; `spctl -a` already returns "rejected". No quarantine
xattr (only com.apple.provenance). Adding mod files breaks the seal but
there is no enforced signature. Rollback: `rm -rf <bundle>/Contents/MacOS/mods`.
### Installed
STS2MCP v0.4.0 -> `SlayTheSpire2.app/Contents/MacOS/mods/`
(staged copy kept in `~/sts2-bot/vendor/`)
Mods load at startup, so the game must be restarted.
## VERSION DRIFT: STS2MCP v0.4.0 is broken on game v0.107.1
### Symptom
GET /api/v1/singleplayer returns HTTP 500 once combat starts:
"error": "Failed to read game state: Method not found:
'Boolean MegaCrit.Sts2.Core.Combat.CombatManager.get_IsPlayPhase()'."
"exception_type": "System.MissingMethodException"
at STS2_MCP.McpMod.BuildBattleState(RunState, CombatRoom)
Menu, map, and character-select states work. COMBAT state fails.
### Root cause
`CombatManager.IsPlayPhase` was removed in game v0.107.1.
Verified: grep -c IsPlayPhase sts2.xml -> 0.
Replacement in v0.107.1 (confirmed in sts2.xml):
* `MegaCrit.Sts2.Core.Combat.PlayerTurnPhase.Play` (per-player)
* `PlayerCombatState.Phase`
* `ActionSynchronizerCombatState.PlayPhase` / `.NotPlayPhase`
* `CombatManager.DebugOnlyGetState().CurrentSide == CombatSide.Player`
### Timeline
* release 0.4.0 2026-05-05 (broken, pre-fix)
* fix commit 55e06485 2026-07-29 PR #123 "Fix game API compatibility with STS2 v0.107"
* no tag after 0.4.0, no CI workflow, no fork ships a prebuilt DLL
### Resolution: build from source
Installed .NET 9 SDK ephemerally via nix (no system changes):
nix eval --raw nixpkgs#dotnet-sdk_9.name -> dotnet-sdk-wrapped-9.0.310
Build (macOS):
cd ~/sts2-bot/vendor/STS2MCP
nix shell nixpkgs#dotnet-sdk_9 --command bash -c '
dotnet build STS2_MCP.csproj -c Release -o out/STS2_MCP \
-p:STS2GameDir="$HOME/Library/Application Support/Steam/steamapps/common/Slay the Spire 2"'
Result: Build succeeded, 0 warnings, 0 errors.
Installed 236,544 B dll (release was 194,560 B).
Old release dll kept at vendor/STS2_MCP.dll.release-0.4.0.
### Notes
* Upstream open issues #131/#132 track v0.111 compatibility. Re-check before
upgrading the game, since the same class of break will recur.
* The csproj resolves the data dir on macOS automatically to
SlayTheSpire2.app/Contents/Resources/data_sts2_macos_arm64 and references
sts2.dll, GodotSharp.dll, 0Harmony.dll with Private=false.
## Confirmed save layout (modded)
~/Library/Application Support/SlayTheSpire2/steam/<steamid>/
settings.save {"mod_settings": {"mods_enabled": true}}
profile.save
profile1/saves/... vanilla progress
modded/profile1/saves/ MODDED progress (isolated)
prefs.save, progress.save, history/
Modded runs never touch vanilla saves.
## Confirmed state shapes
menu (main):
{state_type, menu_screen, message, options[]}
character_select:
{state_type, menu_screen, message, characters[]{name,id,locked,hp,gold,energy,
description,starting_relics[],starting_deck[],total_cards,total_relics,total_potions},
options[]{name,enabled}}
Only IRONCLAD unlocked on a fresh profile. No seed option in standard SP
(docs: supplying seed here errors).
tutorial_prompt:
{state_type, menu_screen, message, options[]{name,enabled}} -> choose 'no'
map:
{state_type, map{visited,next_options[]{index,col,row,type,leads_to},nodes[]},
run{act,floor,ascension},
player{character,hp,max_hp,block,gold,status[],relics[],potions[],max_potion_slots}}
No seed field is exposed in run state.
combat: BLOCKED until the rebuilt dll loads.
# MILESTONE: bot reached the Act 1 boss
## First full run result (run history file)
~/Library/Application Support/SlayTheSpire2/steam/<id>/modded/profile1/saves/history/<ts>.run
win : false
killed_by_encounter : "ENCOUNTER.VANTOM_BOSS"
seed : "JUS59Z32HF"
run_time : 2017 s
build_id : "v0.107.1"
ascension : 0
game_mode : "standard"
was_abandoned : false
acts : 3
schema_version : 9
The bot cleared 15 floors, beat an elite (Byrdonis, 81 HP), healed at rest
sites, and died to the Act 1 boss Vantom on floor 16.
## Evaluation harness: SOLVED
The .run history file IS the eval record. It exposes win/loss, killed_by,
seed, run_time, and build_id. Stage 5 (win rate over seeds) reads this file.
The seed is only visible HERE -- it is not in live state and cannot be set
for standard singleplayer.
## Decision sources (759 loop steps, 5 sessions)
code 141 deterministic: lethal lines, procedure, arithmetic
jev 70 preference: which card, which node, which event option
fallback 35 heuristic: low-confidence recovery, no-Jev mode
lethal lines executed by code: 33
## State shapes discovered (all verified live)
menu {state_type, menu_screen, message, options[],
blocked_options[]{name,enabled,reason,pending_epoch_ids}}
character_select {characters[]{name,id,locked,hp,gold,energy,description,
starting_relics[],starting_deck[],total_cards,...}, options[]}
map {map{visited,next_options[]{index,col,row,type,leads_to},nodes[],
boss{id,name}}, run, player}
monster/elite/boss
{battle{round,turn,is_play_phase,enemies[]{entity_id,combat_id,
name,hp,max_hp,block,status[]{id,name,amount,type,description},
intents[]{type,label,title,description}}},
run{act,floor,ascension},
player{character,hp,max_hp,block,energy,max_energy,hand[],
draw_pile[],draw_pile_count,discard_pile[],exhaust_pile[],
gold,status[],relics[],potions[],max_potion_slots}}
card : {id,name,type,cost(str),star_cost,description,rarity,is_upgraded,
keywords[],index,target_type,can_play,unplayable_reason}
pile card: {name,cost,star_cost,description} <-- NO id/type/index
intent label: damage as a STRING, e.g. "12"; may be "12x2"
rewards {rewards{items[]{index,type,description,[gold_amount],
[potion_id,potion_name,potion_description]}, can_proceed}}
items[] contains ONLY enabled rewards, re-indexed per claim
card_reward {card_reward{cards[], can_skip}} <-- NO deck composition
card_select {card_select{screen_type,prompt,cards[],preview_showing,
preview_cards,can_cancel,can_confirm}}
event {event{in_dialogue,body,options[]{index,title,description,
is_locked,is_proceed,was_chosen,[relic_name,relic_description]}}}
rest_site {rest_site{options[]{index,id,name,description,is_enabled},
can_proceed}} <-- `name`, NOT `title`
treasure {treasure{relics[]{index,id,name,description,rarity},
can_proceed,message}} <-- relics ABSENT while chest opens
shop {shop{items[],can_proceed}}
## Bugs found live and fixed (all were real infinite loops)
1. claim_reward(index=0) forever.
items[] re-indexes on every claim. Must claim RIGHT-TO-LEFT.
2. claim_reward on a potion with all slots full.
Returns "ok" and is SILENTLY DROPPED, so the item never leaves the list.
Fix: discard the weakest potion first (POTION_VALUE table in brain.py).
3. select_card twice on one index.
It TOGGLES on grid screens; the second call deselects and freezes the
screen. Fix: if preview_showing and can_confirm -> confirm_selection.
4. Stale preview: confirm_selection returns ok and changes nothing.
Fix: if the identical card_select state repeats, cancel_selection to reset.
5. rest_site read options[].title, but the field is `name` (+ is_enabled).
Fix applied.
6. treasure claimed a relic before the chest auto-opened.
relics[] is absent during opening; only can_proceed is set.
Fix: wait while relics absent.
7. Acting during transitions.
choose_map_node fired 3x during travel; end_turn during the enemy turn.
Fix: __wait__ on not-in-play-phase, plus an unchanged-state guard in run.py
that waits when the last action LANDED, and re-decides when it was REJECTED.
8. state_type 'unknown'/'overlay' treated as a dead end.
These are transitions. Fix: wait instead of stopping.
## Confidence gate bug (found by measurement)
A fixed confidence floor is wrong when option count varies. Measured:
5 cards, Jev picks Bash at 0.61 with Defend at 0.29 -> confidence 0.50
because confidence measures peakedness: (5*0.61-1)/4 = 0.50.
A 0.55 floor rejected a clear plurality.
Fix: gate a Choice on (top >= 0.45 AND top - runner_up >= 0.20).
Noul has no confidence; gate on |noul-0.5| >= 0.15.
## BLOCKER: Timeline epoch reveal needs a human
After the run ended, the main menu offers only `settings` and `quit`.
blocked_options reports:
{"name":"timeline","enabled":false,
"reason":"manual_epoch_reveal_required",
"pending_epoch_ids":["NEOW_EPOCH"]}
menu_select("timeline") refuses:
"Timeline has obtained epochs that still need to be revealed manually;
not opening Timeline because this game state logs invalid unlock-state
errors when entered through automation"
This is a deliberate guard in the mod, not a bug. The user must open the
Timeline in-game and reveal NEOW_EPOCH by hand. Until then the bot cannot
start a new run.
## Test suite
test_facts.py: 29 assertions, all passing, including the exact case where
jev-1.13.0 answered "lethal available" 0.79 YES when the true answer was NO.

174
docs/research/01-game-engine-and-mod-surface.md Normal file
View file

@ -0,0 +1,174 @@
# 01 — Game engine and mod surface
Measured on game build `v0.107.1` (commit `59260271`), macOS arm64, Steam.
## Engine
Slay the Spire 2 is a **Godot 4.5.1 (.NET)** game, not Java like the original.
This matters: the gameplay code is plain managed C# IL, so it can be read
directly with no decompiler.
| Item | Value |
|---|---|
| Engine | Godot 4.5.1 (.NET build) |
| Runtime | .NET 9.0.7, self-contained `osx-arm64` |
| Root namespace | `MegaCrit.Sts2.*` (2,751 types) |
| Bundle id | `com.megacrit.SlayTheSpire2` |
## Layout inside the app bundle
```
SlayTheSpire2.app/Contents/
MacOS/
Slay the Spire 2 179 MB Mach-O universal (x86_64 + arm64)
mods/ <- mods load from here
Frameworks/
libfmod.dylib, libfmodstudio.dylib
libGodotFmod.macos.template_release.framework
libspine_godot.macos.template_release.framework
libSentry.dylib
Resources/
Slay the Spire 2.pck 1.9 GB Godot pack
data_sts2_macos_arm64/ .NET assemblies + runtime
sts2.dll 9.3 MB main game assembly
sts2.xml 5.3 MB XML docs, 19,635 members
GodotSharp.dll, 0Harmony.dll, MonoMod.Backports.dll,
Steamworks.NET.dll, Sentry.dll, ...
data_sts2_macos_x86_64/ same, other arch
```
### The free win: `sts2.xml`
`sts2.xml` is the .NET XML documentation file, shipped un-stripped. It gives
**19,635 documented members** with prose summaries. Example, verbatim:
> `ModInitializerAttribute` — *"Declares a class as the main entry point for
> the mod. If this is present, then upon loading the mod, we'll call the method
> named `initializerMethod` within the class. Otherwise, we'll create a harmony
> instance for the mod and call `Harmony.PatchAll`."*
This removes most of the guesswork from modding. Query it with
`~/sts2-re/docdump.py` or any XML parser.
### Harmony and MonoMod ship in the game
`0Harmony.dll` and `MonoMod.*.dll` are present in the shipping build. That is
a strong signal that runtime patching is a supported path, not an accident.
## Official mod loader
The game has first-class mod support. Namespace `MegaCrit.Sts2.Core.Modding`.
| Type | Purpose |
|---|---|
| `ModManager` | Discovers and loads mods |
| `ModInitializerAttribute` | Declares a mod entry point |
| `ModManifest` | The JSON manifest |
| `ModHelper` | Content registration and hook subscription |
| `ModSource` | `ModsDirectory` or `SteamWorkshop` |
| `SettingsSaveMod` | Per-mod enable/disable and manual load order |
`ModManager.Initialize` is documented as loading mods *"from the 'mods'
directory next to the executable, as well as Steam workshop files"* and is
*"called as early as possible in the game's initialization process"*.
**Consequence:** mods load at process start. Changing a mod requires a full
restart. "Save and quit" to the main menu is not enough.
### Mod directories
| Source | Path |
|---|---|
| Local (macOS) | `SlayTheSpire2.app/Contents/MacOS/mods/` |
| Steam Workshop | `Steam/steamapps/workshop/content/2868840/` |
### Manifest schema (verified from a working mod)
```json
{
"id": "STS2_MCP",
"name": "STS2 MCP",
"author": "kunology",
"description": "MCP server bridge for Slay the Spire 2",
"version": "0.4.0",
"has_pck": false,
"has_dll": true,
"affects_gameplay": false
}
```
Note **snake_case** in JSON (`affects_gameplay`, `has_dll`, `has_pck`), which
differs from the C# property names (`affectsGameplay`). The manifest filename
must match the mod id: `STS2_MCP.json` for id `STS2_MCP`.
### Mod loading, from the live log
```
Found mod manifest file .../mods/STS2_MCP.json
Mods have been re-sorted because we detected a change or dependency order was broken.
[WARN] Mod STS2_MCP does not declare min game version. Assuming that it is supported.
Loading assembly DLL .../mods/STS2_MCP.dll
Calling initializer method of type STS2_MCP.McpMod for STS2_MCP, Version=1.0.0.0
Finished mod initialization for 'STS2 MCP' (STS2_MCP).
--- RUNNING MODDED! --- Loaded 1 mods (1 total)
[Sentry.NET] Is running modded
```
`minGameVersion` is optional but its absence produces a warning. It is worth
declaring in any mod we write.
### Consent flow
`NConfirmModLoadingPopup` is documented as *"Vertical popup used to get player
confirmation before loading mods. Renders above the capstone screens (above top
bar)."* It appears **over the main menu, at launch**, and only when mods are
present. Consent persists as `mod_settings: {"mods_enabled": true}` in
`settings.save`.
There is also a `nomods` command-line argument that makes `ModManager` skip
initialization entirely (`ModManagerState.Skipped`).
## Save layout
Modded and vanilla progress are **isolated**:
```
~/Library/Application Support/SlayTheSpire2/steam/<steamid>/
settings.save {"mod_settings": {"mods_enabled": true}, ...}
profile.save
profile1/saves/ vanilla progress
modded/profile1/saves/ MODDED progress
prefs.save
progress.save unlocks, card/enemy/encounter stats
current_run.save the run in progress
history/<ts>.run completed run records
```
Modded runs never touch vanilla saves. This was confirmed by the log line
`Profile-scoped data path initialized: user://steam/<id>/modded/profile1`.
## Bundle signing
The app is **ad-hoc signed** (`Signature=adhoc`, `flags=0x10002(adhoc,runtime)`)
and `spctl -a` already returns `rejected`. There is no quarantine attribute
(only `com.apple.provenance`). Adding files to `Contents/MacOS/` breaks the
sealed-resource count but there is no enforced signature, and Steam launches
the binary directly.
**Do not run Steam's "Verify integrity of game files"** — it deletes the mods
directory, because those files are not in the depot manifest.
## Useful namespaces for a bot
| Namespace | Why it matters |
|---|---|
| `Core.Modding` | Official mod API |
| `Core.AutoSlay` | MegaCrit's own auto-play bot, shipped in the build |
| `Core.DevConsole.ConsoleCommands` | ~40 built-in debug commands |
| `Core.GameActions` | Command pattern for game actions |
| `Core.Runs` / `Core.Combat` | Run and combat state |
| `Core.Saves` | Source-generated JSON serialization |
`AutoSlayer.Start(seed, ...)` runs a complete automated game with handlers for
every screen and room. It is MegaCrit's smoke-test harness and is the best
available reference for driving the game programmatically.

199
docs/research/02-system-one-jev.md Normal file
View file

@ -0,0 +1,199 @@
# 02 — System One / Jev
Everything here is measured against `jev-latest` → `jev-1.13.0` from
`https://api.typesafe.ai/v1/systemone`.
## What Jev is
TypeSafe states it plainly:
> "System One is TypeSafe's model for building AI-powered software, not agents.
> It does not generate code or choose its own next action."
It is a **calibrated classifier**. It evaluates a `state` and answers typed
questions about it. It cannot plan, cannot call tools, and cannot invent values
outside the option set you supply.
| Primitive | Shape | Returns |
|---|---|---|
| `Choice` | pick 1 of ≤ 255 options | `choice`, `probabilities`, `confidence` |
| `Score` | position on ordered levels | `score`, `legend`, `probabilities`, `confidence` |
| `Noul` | yes/no | `noul` (probability only, **no confidence**) |
Documented properties: questions in one call are evaluated **independently and
in parallel**. One answer is never hidden context for another.
## Measured latency — 7× the documented figure
The docs say "most queries complete in about 100 ms". Measured end-to-end from
this machine:
| Request | Time |
|---|---|
| Minimal (1 short noul) | 0.73 s |
| Minimal × 5 repeats | 0.73, 0.73, 0.74, 0.80, 0.76 s |
| Full combat state, 3 questions | 0.90 s |
| `GET /v1/models` TTFB | 0.68 s (connect 0.22 s) |
The floor is **~0.73 s**, dominated by network RTT plus server TTFB.
**The useful consequence:** a 1-question call and a 3-question call over a full
combat state differ by only 0.17 s. State size and question count are nearly
free. **Batch every question for a state into one call.**
## Cost
| | |
|---|---|
| Input | $0.042 per Mtok |
| Output | free |
| Rate limit | 250k tok/s, 1,200 req/min |
| Context | 64k total; 32k for `state` + longest question |
| Input type | text only (string, JSON object, array) |
For comparison, the STS2MCP README reports a full run costs a frontier LLM
about **8M tokens** (~$20–40). The same volume through Jev is roughly
**$0.08–0.35 per run**.
## THE CRITICAL FINDING: Jev cannot do arithmetic
Ground truth: `energy 3`, hand `Strike(1 cost, 6 dmg) ×3` plus
`Bash(2 cost, 8 dmg)`, target on `19 HP`. Maximum reachable damage is **18**,
so lethal is **NO**.
| Question | Jev answered | Verdict |
|---|---|---|
| Max damage bucket | `18_to_23` @ 0.73 | **Correct** |
| Is lethal available? | `0.79` | **Wrong** |
Jev bucketed the magnitude correctly but failed the threshold comparison of 18
versus 19 — **and reported 0.79 confidence on the wrong answer**.
Two conclusions:
1. Jev is decent at *approximate magnitude*, poor at *exact comparison*.
2. **Confidence gating cannot protect against arithmetic errors.** A
0.79-confidence wrong answer passes any sane threshold.
This is why `facts.py` computes every sum, comparison, and threshold, and hands
Jev only conclusions (`"lethal_available": true`). This is a regression test:
see `test_facts.py` case 1.
The jaggedness page confirms the general shape: Jev "does not count reliably",
"is not a calculator", degrades with indirection, and suffers "context rot"
when the state carries irrelevant detail.
## The confidence gate trap
**A fixed confidence floor is wrong when the option count varies.**
Measured: 5 cards offered. Jev picks `Bash` at **0.61**, with `Defend` at 0.29.
Reported `confidence` is **0.50**.
Why: confidence measures *peakedness*. For 5 options the documented formula is
`(count × peak − 1) / (count − 1)` = `(5 × 0.61 − 1) / 4` = 0.50.
A 0.55 floor rejected a clear plurality and fell back to a worse heuristic.
With 5 options, 0.50 is a strong plurality; with 2 options, 0.50 is a coin flip.
**Fix:** gate a `Choice` on margin over the runner-up, which is scale-free:
```python
top = probabilities[choice]
runner = max(v for k, v in probabilities.items() if k != choice)
act = top >= 0.45 and (top - runner) >= 0.20
```
`Noul` has no confidence field, so gate it on distance from 0.5:
```python
act = abs(noul - 0.5) >= 0.15 # act when noul >= 0.65 or <= 0.35
```
## Design rules that follow from the above
1. **Never** route an arithmetic comparison through Jev. Compute it in code.
2. Send **conclusions and buckets**, not raw numbers to be compared.
3. Keep `state` small and relevant — context rot is real and measurable.
4. Batch all questions for one state into **one** call.
5. Questions in one call are independent; if Q2 needs Q1's answer, make a
second call.
6. Gate `Choice` on margin, not confidence.
7. Treat a low-confidence answer as a reason to fall back, not to guess.
## Credential handling
The key lives in a sops-nix managed file, **not** in the shell environment:
```
~/.config/secrets/global-env/TYPESAFEAI_API_KEY
-> ~/.config/sops-nix/secrets/TYPESAFEAI_API_KEY
```
`jev.py` reads it at runtime and never logs it. `JevClient.__repr__` prints
`key=REDACTED`, so the secret cannot leak through a traceback or log line.
---
# Session 2 additions
## Option count dilutes a Choice — measured twice
### Shop, 14 candidates
A shop offered 14 affordable items in one `Choice`. Jev's top pick scored only
**0.26** (runner 0.17, margin 0.09) — the probability mass spread across all
fourteen. The margin gate correctly rejected it, so the bot would always leave
the shop with gold unspent.
**Fix:** use the documented **re-ranking** pattern — one *absolute* `Noul` per
candidate, then take the argmax in code. Absolute judgements do not dilute as
the candidate count grows.
### Same problem, smaller: 5 cards
Already covered above. With 5 options, `confidence` is `(5 × top − 1) / 4`, so
a clear plurality reads as 0.50. Gate on margin, not confidence.
## The question framing matters more than the threshold
Asking Jev to weigh value against a **number** degrades its judgement, exactly
as the jaggedness page predicts. Measured on the same shop, same state:
| Framing | Spread across candidates | Top item |
|---|---|---|
| "worth its **72 gold** price for this deck?" | **0.28** | Bag of Preparation 0.51 |
| "would this make this deck stronger?" | **0.48** | Bag of Preparation 0.67 |
| "does this fit what this deck is doing?" | 0.51 | Ashen Strike 0.71 |
| "improve more than it dilutes?" | 0.42 | Bag of Preparation 0.65 |
Putting the price in the question **halved the spread** and pulled the top item
below any usable threshold.
**Rule:** filter affordability in code, keep the price in the `state` for
context, and keep it **out of the question**. Ask about deck fit only.
## Risk is not a preference, and Jev is bad at spotting danger
On an event offering "Keep Deciphering" and "Lose Everything":
| Option | "Does this risk losing the run?" |
|---|---|
| "Lose Everything" | **0.46** |
| "Keep Deciphering" | 0.52 |
| "Stop" | 0.38 |
The model ranked the run-ending option as less risky than a moderate one. Do
not use a model judgement to detect danger. Use deterministic keyword matching
plus a stricter confidence gate. See [05](05-failure-modes.md) §10.
## Confirmed working patterns
| Pattern | Where used | Result |
|---|---|---|
| Batch all questions for one state in one call | combat | 0.73 s for 1 question, 0.90 s for 3 |
| Absolute `Noul` per candidate, argmax in code | shop | Works where a 14-way Choice failed |
| Compute the hard fact in code, let Jev pick | lethal, potions | 33 lethal lines executed without the model |
| Gate on margin, not confidence | every `Choice` | Fixed a rejected-correct-answer bug |
| Keep arithmetic out of the question | shop | Doubled the usable spread |
| Deterministic safety net for danger | events | Caught what the model missed |

168
docs/research/03-sts2mcp-interface.md Normal file
View file

@ -0,0 +1,168 @@
# 03 — STS2MCP interface
A community mod by `kunology` that exposes the running game over a local HTTP
API. Repository: `github.com/Gennadiyev/STS2MCP`, MIT.
It is the actuator for this project: it reads state and performs one action at
a time. It does **not** alter gameplay (`affects_gameplay: false`).
## Endpoints
Base URL `http://localhost:15526`, no authentication.
| Method | Path | Purpose |
|---|---|---|
| GET | `/api/v1/singleplayer?format=json\|markdown` | Read state |
| POST | `/api/v1/singleplayer` | Perform one action |
| GET | `/api/v1/multiplayer` | Read multiplayer state |
| POST | `/api/v1/multiplayer` | Multiplayer action |
| GET | `/api/v1/profile` | Persistent profile progress |
| GET | `/api/v1/compendium` | Compendium-shaped progress |
| GET | `/api/v1/wiki?query=&item_type=&limit=` | Fuzzy card/relic lookup |
| GET/POST | `/api/v1/profiles` | List / switch / delete profile slots |
All POST bodies carry an `"action"` field. Responses carry
`{"status": "ok"|"error", "message": ...}`.
## `state_type` is the action enumerator
Every response carries `state_type`, and each type has a small fixed action
set. **This is the action space.** There is no need to write one.
| `state_type` | Legal actions |
|---|---|
| `menu`, `game_over` | `menu_select` |
| `monster` / `elite` / `boss` | `play_card`, `use_potion`, `end_turn` |
| `hand_select` | `combat_select_card`, `combat_confirm_selection` |
| `rewards` | `claim_reward`, `proceed` |
| `card_reward` | `select_card_reward`, `skip_card_reward` |
| `map` | `choose_map_node` |
| `event` | `choose_event_option`, `advance_dialogue` |
| `rest_site` | `choose_rest_option`, `proceed` |
| `shop`, `fake_merchant` | `shop_purchase`, `proceed` |
| `treasure` | `claim_treasure_relic`, `proceed` |
| `card_select` | `select_card`, `confirm_selection`, `cancel_selection` |
| `bundle_select` | `select_bundle`, `confirm_bundle_selection`, `cancel_bundle_selection` |
| `relic_select` | `select_relic`, `skip_relic_selection` |
| `crystal_sphere` | `crystal_sphere_set_tool`, `_click_cell`, `_proceed` |
| `overlay`, `unknown` | none (transitions, or manual) |
`use_potion` and `discard_potion` work in any state where potions are
reachable, including outside combat.
## VERSION DRIFT — release 0.4.0 is broken on this build
### Symptom
`GET /api/v1/singleplayer` returns **HTTP 500** as soon as combat starts:
```json
{
"error": "Failed to read game state: Method not found:
'Boolean MegaCrit.Sts2.Core.Combat.CombatManager.get_IsPlayPhase()'.",
"exception_type": "System.MissingMethodException",
"stack_trace": " at STS2_MCP.McpMod.BuildBattleState(RunState, CombatRoom)"
}
```
Menu, character select, map, rewards, and events all work. **Only combat
fails** — the one state that matters most.
### Root cause
`CombatManager.IsPlayPhase` was removed in game `v0.107.1`. Verified:
```bash
grep -c IsPlayPhase sts2.xml # -> 0
```
Replacements present in `v0.107.1`:
- `MegaCrit.Sts2.Core.Combat.PlayerTurnPhase.Play` (per-player)
- `PlayerCombatState.Phase`
- `ActionSynchronizerCombatState.PlayPhase` / `.NotPlayPhase`
- `CombatManager.DebugOnlyGetState().CurrentSide == CombatSide.Player`
### Timeline
| Event | Date |
|---|---|
| Release `0.4.0` (broken) | 2026-05-05 |
| Fix commit `55e06485`, PR #123 | 2026-07-29 |
| Newer tag | none |
| CI build artifact | none |
| Fork shipping a prebuilt DLL | none (checked 12 forks) |
The fix exists in source but was never released.
### Resolution
Build from upstream `main`. The fix is present and documented in the code:
```csharp
// STS2 v0.107 removed CombatManager.IsPlayPhase. The turn phase now lives per-player on
// PlayerCombatState.Phase
internal static bool IsPlayPhase(Player? player)
{
if (!IsPlayerSideTurn()) return false;
return player?.PlayerCombatState?.Phase == PlayerTurnPhase.Play;
}
```
```bash
nix shell nixpkgs#dotnet-sdk_9 --command bash -c '
dotnet build STS2_MCP.csproj -c Release -o out/STS2_MCP \
-p:STS2GameDir="$HOME/Library/Application Support/Steam/steamapps/common/Slay the Spire 2"'
```
`Build succeeded. 0 Warning(s) 0 Error(s).` Result: 236,544 B DLL, replacing
the 194,560 B release. The release DLL is kept at
`vendor/STS2_MCP.dll.release-0.4.0`.
**Lesson:** the game moves faster than the mod's releases. Upstream open issues
#131 and #132 already track `v0.111` compatibility. Expect this class of break
to recur after any game update. Pin the mod build to the game build.
## Known doc drift inside the mod
`AGENTS.md` in the repo uses MCP **tool** names; `docs/raw-simplified.md` uses
the HTTP **action** names. They do not match:
| AGENTS.md | HTTP action |
|---|---|
| `combat_end_turn` | `end_turn` |
| `event_choose_option` | `choose_event_option` |
| `rewards_pick_card` | `select_card_reward` |
| `rest_choose_option` | `choose_rest_option` |
| `proceed_to_map` | `proceed` |
Trust `raw-simplified.md` and the live API for direct HTTP use.
## Hard blocker: Timeline cannot be automated
After a run ends with a pending epoch, the mod refuses:
```json
{"status":"error",
"error":"Timeline has obtained epochs that still need to be revealed manually;
not opening Timeline because this game state logs invalid unlock-state
errors when entered through automation",
"pending_epoch_ids":["NEOW_EPOCH"],
"manual_action_required":true}
```
This is a **deliberate guard**, not a bug. Entering Timeline through automation
would corrupt unlock state. A human must open the Timeline and reveal the
epoch. Until then the main menu offers only `settings` and `quit` and no run
can be started.
## Mod config
The mod writes `mods/STS2_MCP.conf`:
```json
{ "port": 15526 }
```
It also injects an **"Instant Mode"** checkbox into the game settings, which
shortens animations. Worth enabling for batch runs.

430
docs/research/04-state-shapes.md Normal file
View file

@ -0,0 +1,430 @@
# 04 — State shapes
Every shape below was captured from a **live game**, not read from docs. Keys
are exact. Where a field is missing in some states, that is called out, because
each absence caused a real bug.
Captured from `GET /api/v1/singleplayer?format=json` on game `v0.107.1`.
---
## `menu`
```json
{
"state_type": "menu",
"menu_screen": "main",
"message": "Main menu.",
"options": ["singleplayer", "multiplayer", "compendium", "timeline", "settings", "quit"],
"blocked_options": [
{"name": "timeline", "enabled": false,
"reason": "manual_epoch_reveal_required",
"pending_epoch_ids": ["NEOW_EPOCH"]}
]
}
```
`options` entries are **plain strings** here. On other screens they are objects
`{"name": ..., "enabled": ...}`. Handle both.
Observed `menu_screen` values: `main`, `character_select`, `tutorial_prompt`.
Mode selection (after the first epoch unlock) offers
`standard`, `daily`, `custom`, `back`. `daily` and `custom` are the seeded
modes; standard singleplayer exposes no seed.
## `character_select`
```json
{
"state_type": "menu",
"menu_screen": "character_select",
"message": "Select a character.",
"characters": [{
"name": "The Ironclad", "id": "IRONCLAD", "locked": false,
"hp": 80, "gold": 99, "energy": 3,
"description": "Ironclad cards will now appear in rewards and shops.",
"starting_relics": [{"name": "Burning Blood", "description": "At the end of combat, heal 6 HP."}],
"starting_deck": ["Strike","Strike","Strike","Strike","Strike",
"Defend","Defend","Defend","Defend","Bash"],
"total_cards": 87, "total_relics": 8, "total_potions": 3
}],
"options": [{"name": "IRONCLAD", "enabled": true}, {"name": "SILENT", "enabled": false},
{"name": "confirm", "enabled": true}, {"name": "embark", "enabled": true},
{"name": "back", "enabled": true}]
}
```
All five characters are listed, with `locked` set per profile. `starting_deck`
is the cheapest reliable source of the initial deck for a deck tracker.
## `monster` / `elite` / `boss`
```json
{
"state_type": "monster",
"battle": {
"round": 1, "turn": "player", "is_play_phase": true,
"enemies": [{
"entity_id": "NIBBIT_0", "combat_id": 1, "name": "Nibbit",
"hp": 44, "max_hp": 44, "block": 0,
"status": [{"id": "TERRITORIAL_POWER", "name": "Territorial", "amount": 1,
"type": "Buff", "description": "At the end of Byrdonis's turn, it gains 1 Strength.",
"keywords": [{"name": "Strength", "description": "..."}]}],
"intents": [{"type": "Attack", "label": "12", "title": "Aggressive",
"description": "This enemy intends to Attack for 12 damage."}]
}]
},
"run": {"act": 1, "floor": 1, "ascension": 0},
"player": {
"character": "The Ironclad",
"hp": 80, "max_hp": 80, "block": 0,
"energy": 3, "max_energy": 3,
"hand": [{
"id": "STRIKE_IRONCLAD", "name": "Strike", "type": "Attack",
"cost": "1", "star_cost": null,
"description": "Deal 6 damage.", "rarity": "Basic", "is_upgraded": false,
"keywords": [], "index": 1, "target_type": "AnyEnemy",
"can_play": true, "unplayable_reason": null
}],
"draw_pile_count": 5, "discard_pile_count": 0, "exhaust_pile_count": 0,
"draw_pile": [{"name": "Defend", "cost": "1", "star_cost": null, "description": "Gain 5 Block."}],
"discard_pile": [], "exhaust_pile": [],
"gold": 99, "status": [], "relics": [], "potions": [], "max_potion_slots": 3
}
}
```
Critical details:
- **`cost` is a STRING** (`"1"`), not an int. Parse it. Some cards may be `"X"`.
- **`intents[].label` carries the damage as a STRING** (`"12"`), and may be
`"12x2"` for multi-hit. This is the authoritative number for incoming damage.
- **Pile cards carry only `{name, cost, star_cost, description}`** — no `id`,
no `type`, no `index`. Deck composition must therefore be counted **by name**,
not by type. This caused a bug.
- `intents` is absent when the enemy has no next move.
- `status` entries are powers: `{id, name, amount, type, description, keywords}`.
- `target_type` values seen: `Self`, `AnyEnemy`.
- `unplayable_reason` is set when `can_play` is false.
## `rewards`
```json
{
"state_type": "rewards",
"rewards": {
"items": [
{"index": 0, "type": "gold", "description": "19 Gold", "gold_amount": 19},
{"index": 1, "type": "potion", "description": "Energy Potion",
"potion_id": "ENERGY_POTION", "potion_name": "Energy Potion",
"potion_description": "Gain [ironclad_energy_icon.png][ironclad_energy_icon.png]."}
],
"can_proceed": true
},
"run": {"act": 1, "floor": 1, "ascension": 0},
"player": { "...": "same as combat, but no hand/energy" }
}
```
- `items[]` contains **only enabled rewards**, re-indexed from 0 on every claim.
Claim **right-to-left**, or index 0 is reclaimed forever.
- Reward `type` values: `gold`, `potion`, `relic`, `card`, `special_card`.
- `items` can be `[]` while `can_proceed` is true. Then the correct action is
`proceed`.
## `card_reward`
```json
{
"state_type": "card_reward",
"card_reward": {
"cards": [{"id": "SETUP_STRIKE", "name": "Setup Strike", "type": "Attack",
"cost": "1", "star_cost": null,
"description": "Deal 7 damage. Gain 2 Strength this turn.",
"rarity": "Common", "is_upgraded": false, "keywords": [], "index": 0}],
"can_skip": true
},
"run": {...}, "player": {...}
}
```
**The deck is NOT exposed here.** Deck-building decisions need a composition
snapshot taken from a combat state, which does expose all four piles.
## `card_select`
```json
{
"state_type": "card_select",
"card_select": {
"screen_type": "upgrade",
"prompt": "Choose a card to Upgrade.",
"cards": [{"id": "STRIKE_IRONCLAD", "name": "Strike", "type": "Attack",
"cost": "1", "description": "Deal 6 damage.",
"rarity": "Basic", "is_upgraded": false, "index": 0}],
"preview_showing": true,
"preview_cards": [
{"name": "Strike", "description": "Deal 6 damage.", "is_upgraded": false},
{"name": "Strike+", "description": "Deal 9 damage.", "is_upgraded": true}
],
"can_cancel": true,
"can_confirm": true
}
}
```
`preview_cards` is a before/after pair — useful for showing Jev what the
upgrade would actually do. `screen_type` values include `upgrade`.
**`select_card` TOGGLES on grid screens.** Calling it twice on one index
deselects and freezes the screen. When `preview_showing` is true and
`can_confirm` is true, the correct action is `confirm_selection`.
## `event`
```json
{
"state_type": "event",
"event": {
"in_dialogue": false,
"body": "…event text…",
"options": [{"index": 0, "title": "…", "description": "…",
"is_locked": false, "is_proceed": false, "was_chosen": false,
"relic_name": "…", "relic_description": "…",
"keywords": []}]
}
}
```
Ancient events begin with `in_dialogue: true`, which requires
`advance_dialogue` first. Option 0 is frequently locked, which is why a blind
`choose_event_option(index=0)` gets rejected.
## `rest_site`
```json
{
"state_type": "rest_site",
"rest_site": {
"options": [{"index": 0, "id": "…", "name": "Rest",
"description": "…", "is_enabled": true}],
"can_proceed": true
}
}
```
**The field is `name`, not `title`**, plus `id` and `is_enabled`.
## `treasure`
```json
{
"state_type": "treasure",
"treasure": {
"relics": [{"index": 0, "id": "…", "name": "…", "description": "…",
"rarity": "…", "keywords": []}],
"can_proceed": true
}
}
```
The chest **auto-opens**. While it opens, the response is
`{"message": "Opening chest..."}` with **no `relics` key** and no
`can_proceed`. Claiming then is rejected. Wait while `relics` is absent.
## `shop`
```json
{
"state_type": "shop",
"shop": {
"items": [{"index": 0, "category": "…", "price": 0,
"is_stocked": true, "can_afford": true, "name": "…",
"description": "…"}],
"can_proceed": true
}
}
```
Items carry `price`, `is_stocked`, and `can_afford` — everything needed to
decide a purchase without extra arithmetic on the model side.
## `shop`
```json
{
"state_type": "shop",
"shop": {
"items": [
{"index": 0, "category": "card", "price": 72, "is_stocked": true,
"can_afford": true, "on_sale": false,
"card_id": "STOMP", "card_name": "Stomp", "card_type": "Attack",
"card_cost": "3", "card_rarity": "Uncommon",
"card_description": "Deal 12 damage to ALL enemies...", "keywords": []},
{"index": 7, "category": "relic", "price": 199, "is_stocked": true,
"can_afford": true, "relic_id": "BLOOD_VIAL", "relic_name": "Blood Vial",
"relic_description": "At the start of each combat, heal 2 HP."},
{"index": 10, "category": "potion", "price": 48, "is_stocked": true,
"can_afford": true, "potion_id": "BLOCK_POTION", "potion_name": "Block Potion",
"potion_description": "Gain 12 Block."},
{"index": 13, "category": "card_removal", "price": 75, "is_stocked": true,
"can_afford": true}
],
"can_proceed": false
}
}
```
**The name and description fields are category-specific:**
| category | name | description |
|---|---|---|
| `card` | `card_name` | `card_description` |
| `relic` | `relic_name` | `relic_description` |
| `potion` | `potion_name` | `potion_description` |
| `card_removal` | *neither* | *neither* |
Reading `name`/`description` yields `None` for every category. `brain.py`
resolves them via `shop_item_text()`.
A shop can offer **14+ affordable items**. Do not put them all in one `Choice`;
see [02](02-system-one-jev.md).
**`can_proceed` is unreliable here** — measured `false` while `proceed()`
worked. Never wait on it.
## `hand_select`
```json
{
"state_type": "hand_select",
"hand_select": {
"mode": "simple_select",
"prompt": "Choose any number of cards to replace.",
"cards": [],
"selected_cards": [{"index": 0, "name": "Defend"}],
"can_confirm": true
},
"battle": {"...": "..."}, "run": {}, "player": {}
}
```
`cards` is what is **still selectable**; `selected_cards` is what is **already
chosen**. `combat_select_card(card_index)` indexes into the selectable cards,
not the hand. When `cards` is empty, `combat_select_card(0)` fails with
`Card index 0 out of range (0 selectable cards)` — the action is
`combat_confirm_selection`.
## `bundle_select`
```json
{
"state_type": "bundle_select",
"bundle_select": {
"screen_type": "bundle",
"prompt": "Choose a bundle.",
"bundles": [
{"index": 0, "card_count": 3, "cards": [
{"id": "ANGER", "name": "Anger", "type": "Attack", "cost": "0",
"description": "Deal 6 damage. Add a copy of this card into your Discard Pile.",
"rarity": "Common", "is_upgraded": false, "keywords": [], "index": 0}
]}
],
"preview_showing": true,
"preview_cards": [],
"can_cancel": true,
"can_confirm": true
}
}
```
Identical preview semantics to `card_select`. `select_bundle` errors with
"A bundle preview is already open - confirm or cancel it first".
## Still not captured
Documented in `raw-simplified.md` but never observed live, so the field names
are unverified:
`relic_select`, `crystal_sphere`, `fake_merchant`, `game_over`, `overlay`.
`brain.py` parses them defensively for that reason.
---
# Index spaces
**Five actions take an index. They are not the same index space.** This is the
single most dangerous thing in the API, because the parameter names are nearly
identical and a wrong index either hits the wrong card or silently fails.
| Action | Parameter | Indexes into | Notes |
|---|---|---|---|
| `play_card` | `card_index` | `combatState.Hand.Cards[N]` | matches `player.hand[N]` |
| `combat_select_card` | `card_index` | `hand.ActiveHolders[N]` | **selectable cards only** |
| `select_card` | `index` | the grid screen's card holders | matches `card_select.cards[N]` |
| `select_card_reward` | `card_index` | the reward screen's holders | matches `card_reward.cards[N]` |
| `shop_purchase` | `index` | the merchant inventory | matches `shop.items[N]` |
Verified in `McpMod.Actions.cs`:
```csharp
// play_card
var hand = player.PlayerCombatState?.Hand;
var card = hand.Cards[cardIndex];
// combat_select_card
var holders = hand.ActiveHolders;
if (index < 0 || index >= holders.Count)
return Error($"Card index {index} out of range ({holders.Count} selectable cards)");
```
`play_card` and `combat_select_card` look like the same call. They are **not**.
`player.hand` is every card in hand; `ActiveHolders` is only the selectable
subset.
## `hand_select` has two index spaces under one name
```json
{
"cards": [{"index": 0, "name": "Defend"}],
"selected_cards": [{"index": 0, "name": "Defend"}]
}
```
Both are called `index` and they are built from different arrays:
- `cards[]` is built by iterating `hand.ActiveHolders` with a fresh counter, so
`cards[i].index == i` and indexes the **selectable** cards.
- `selected_cards[]` is built by iterating `selectedHolders` with a **separate**
counter, so it indexes an entirely different array.
Passing a `selected_cards` index to `combat_select_card` is wrong. This produced
the observed failure:
```
action rejected: Card index 0 out of range (0 selectable cards)
```
## Indices shift, so the loop must be closed
- Playing a card removes it from hand and **renumbers every later index**.
The mod's own notes say to play right-to-left to keep indices stable, or
re-read between plays.
- Claiming a reward **rebuilds and renumbers** the rewards list. Claim
right-to-left.
- The shop renumbers after a purchase.
**Rule:** observe, act ONCE, observe again. Never precompute an action list.
## The lesson
The original card-selection bug was **not** a wrong index. The index field was
correct. The bug was choosing an index by **list position** while ignoring the
card's own context — `name`, `type`, `rarity`, `is_upgraded` — which was present
in the state the whole time. Since the grid lists basic Strikes first, position
0 was always a Strike, so the bot only ever upgraded Strikes.
Selection must be by **identity**, and the index is only the handle used to
address that identity.

542
docs/research/05-failure-modes.md Normal file
View file

@ -0,0 +1,542 @@
# 05 — Failure modes
Every entry below was observed **live**, caused a real infinite loop or a
stalled run, and has a fix in the code. None are theoretical.
The general lesson: this interface is full of actions that report
`{"status": "ok"}` while doing nothing. **Never trust `ok`. Trust the state.**
---
## 1. `claim_reward(index=0)` forever
`rewards.items[]` is rebuilt and **re-indexed from 0 after every claim**.
Claiming index 0 repeatedly reclaims the same slot forever.
Observed: 65 consecutive rejected `claim_reward(index=0)` calls.
**Fix:** claim right-to-left, `items[-1]`.
## 2. A potion reward with full slots silently vanishes
When all potion slots are full, `claim_reward` on a potion reward returns
`ok` and is **silently dropped**. The item never leaves `items[]`, so the loop
never terminates.
Observed:
```
potions: 3 of max_potion_slots 3
claim_reward(1) -> ok | Claiming reward: potion (Energy Potion)
items now: [gold, potion] <- unchanged
claim_reward(0) -> ok | Claiming reward: gold (19)
items now: [potion] <- re-indexed
```
**Fix:** if the last item is a potion and `len(potions) >= max_potion_slots`,
discard the weakest potion first, then claim. Weakness comes from a small
`POTION_VALUE` table in `brain.py`.
## 3. `select_card` twice on one index freezes the screen
On grid screens `select_card` **toggles**. The second call deselects, so the
state never changes and the loop stalls.
Observed: `select_card(index=0)` twice, then 10 unchanged reads.
**Fix:** when `preview_showing` and `can_confirm` are true, the action is
`confirm_selection`, never another `select_card`.
## 4. A stale preview makes `confirm_selection` a no-op
`confirm_selection` returns `ok` while changing nothing, when the preview was
left over from a desynchronised state.
Observed:
```
confirm_selection -> ok "Confirming selection from preview"
state_type before/after: card_select -> card_select
sig changed: False
```
**Fix:** if the identical `card_select` state is seen twice in a row, the
confirm did not take effect. Send `cancel_selection` to reset, then re-select.
Verified working:
```
cancel_selection -> preview False
select_card(0) -> preview True
confirm_selection-> state_type becomes 'event'
```
## 5. `rest_site` options use `name`, not `title`
Reading `options[].title` yields `None` for every option, so the handler
concluded there were no options and fell through to a rejected `proceed`.
**Fix:** read `name` (plus `id`), and honour `is_enabled`.
## 6. `treasure` claims a relic before the chest opens
The chest auto-opens. During opening the response has no `relics` key and no
`can_proceed`, so `claim_treasure_relic` is rejected repeatedly.
Observed: 4 consecutive rejections.
**Fix:** wait while `relics` is absent; claim only when present.
## 7. Acting during transitions
The loop re-decided faster than the game animated:
- `choose_map_node` fired **3 times in a row** during a single travel.
- `end_turn` fired repeatedly during the enemy turn.
**Fix, two parts:**
- `__wait__` when `battle.is_play_phase` is false.
- An **unchanged-state guard** in `run.py`: if the state signature repeats and
the **last action succeeded**, wait. If the last action was **rejected**,
re-decide instead, so a different action can be tried.
The second condition matters: waiting on a rejected action would prevent
recovery and stall until the stuck counter fires.
## 8. `unknown` and `overlay` treated as dead ends
Both are transitions, not terminal states. Returning "stop" ended the session
while the game was mid-transition into combat.
**Fix:** wait and re-observe. The unchanged-state guard bounds this, so a
genuine dead end still stops after 10 reads.
## 9. The confidence gate rejected a correct answer
Not a loop, but a wrong decision, and worth recording because the bug was in
**our** gate rather than in the model.
Measured: 5 cards offered. Jev picked `Bash` at **0.61** probability with
`Defend` at 0.29, and reported `confidence 0.50`. A fixed 0.55 floor rejected
it and fell back to a heuristic that chose to block instead.
`confidence` measures peakedness, so it falls as the option count rises:
`(5 × 0.61 − 1) / 4 = 0.50`.
**Fix:** gate a `Choice` on margin over the runner-up
(`top >= 0.45 and top - runner >= 0.20`), which is scale-free. See
[02](02-system-one-jev.md).
---
## Checklist for a new `state_type`
1. Capture the real shape. Do not code from documentation.
2. Check which fields are **absent** in some states.
3. Ask: what does this action return when it is a no-op?
4. Prefer `__wait__` over stopping for anything that looks transitional.
5. Add a guard so a repeated identical state does not loop forever.
## Checklist for a new action
1. Does it report `ok` on a no-op? (Most do.)
2. Does it re-index the collection it acts on?
3. Is there a precondition the state exposes (`can_confirm`, `can_proceed`,
`is_stocked`, `can_afford`) that should be checked first?
4. Does it toggle?
---
# Session 2 additions
## 10. An event option ended the run, and the gate allowed it
Measured on a deciphering event:
```
[102] jev chose Keep Deciphering conf=0.28
[104] jev chose Lose Everything conf=0.49
```
"Lose Everything" **set the player's max HP to 1**. The margin gate passed it,
because the gate only measured how *decisive* the answer was, never how
*consequential* the action was.
**Fix, in two parts:**
1. A stricter gate for events: `top >= 0.60 AND margin >= 0.30`.
2. A deterministic safety net. When the model is not confident, choose the
option with the lowest `event_safety_rank`, which counts risk words
("everything", "keep", "continue", "gamble", "lose") minus stop words
("stop", "leave", "refuse", "decline", "take what").
**A "does this risk losing the run?" Noul was tried and REMOVED.** Measured on
the same event:
| Option | Risk Noul |
|---|---|
| "Lose Everything" | **0.46** |
| "Keep Deciphering" | **0.52** |
| "Stop" | 0.38 |
It ranked the run-ending option as *less* risky than a moderate one. A
misleading signal is worse than no signal, so danger is detected by keywords
instead. Keep the model for the confident case; use code for the dangerous one.
## 11. `hp=1/1` was reported as "healthy"
An effect reduced max HP to 1. `_hp_bucket` bucketed by percentage alone, so
`1/1` was 100% and returned `healthy`. The bot walked into a normal fight at
1 HP and died.
**Fix:** absolute HP is now part of the bucket. `hp <= 5` is always `critical`.
```python
def _hp_bucket(pct, hp=None):
if hp is not None and hp <= 5:
return HP_CRITICAL
...
```
## 12. `hand_select` blindly selected index 0
`combat_select_card(index=0)` failed with:
```
Card index 0 out of range (0 selectable cards)
```
`hand_select` has `cards` (still selectable) and `selected_cards` (already
chosen). When `cards` is empty the only useful action is
`combat_confirm_selection`.
**Fix:** confirm when nothing remains selectable; otherwise give up basic
Strikes first, then Defends.
## 13. `bundle_select` has the same preview trap as `card_select`
```
A bundle preview is already open - confirm or cancel it first
```
Same shape (`preview_showing`, `can_confirm`) and same fix: confirm when a
preview is showing, and reset with `cancel_bundle_selection` if the identical
state repeats.
## 14. Error text lives in `error`, not `message`
`ActionResult` read only `message`, so every rejection printed as
`action rejected:` with nothing after it. This hid four separate bugs for a
whole session.
**Fix:** `message = data.get("message") or data.get("error") or ""`.
**Lesson:** make failures loud before chasing them. A blank error message is
worse than no error handling.
## 15. `can_proceed` is not reliable
For shops, `shop.can_proceed` was `false` while `proceed()` worked and moved the
game to the map. Waiting on that flag stalls forever.
**Fix:** do not gate an exit on `can_proceed`. Attempt `proceed` and let the
rejection counter bound the retries.
## 16. The unchanged-state guard was count-based
A boss death animation plus the rewards transition exceeded 10 reads, so the
guard declared STUCK while the game was still animating.
**Fix:** the guard is now time-based (`--stuck-seconds`, default 25 s).
## 17. Transient rejections are normal
`proceed` at a rest site right after a heal is rejected for a moment and then
succeeds. The retry loop was too impatient.
**Fix:** on rejection, back off 3x the normal pause, and allow 6 attempts.
---
# Session 3 — the programmatic audit
Prompted by observing that the bot "was only upgrading common attack cards".
These are usage bugs, not decision-quality issues, and are listed separately
from the tuning items.
## 18. `card_select` fallback was hardcoded to `cards[0]`
The upgrade screen fell back to the first card in the list, and the list is
ordered with basic Strikes first. So every fallback upgraded a Strike.
Measured across runs:
```
[156] [fallback] select_card(index=0) # select the first card
[263] [fallback] select_card(index=0) # low confidence; select the first
[378] [fallback] select_card(index=0) # low confidence; select the first
```
The fallback fired often because a single `Choice` over a 13+ card deck
dilutes, exactly like the shop.
**Fix, two parts:**
1. Re-ranking: one absolute `Noul` per card, argmax in code.
2. Screen-aware deterministic fallback (`upgrade_rank`, `removal_rank`) that
never picks index 0 blindly:
- upgrade prefers a non-basic card, then Bash, then Strikes, then Defends,
and never an already-upgraded card
- remove/transform invert the order: shed basic Strikes and Defends first,
and never target an upgraded card
Verified live after the fix:
```
[197] jev chose Perfected Strike (noul=0.67)
[205] jev chose Bludgeon (noul=0.66)
[212] jev chose Rampage (noul=0.67)
[303] jev chose Bash (noul=0.62)
```
## 19. `hand_select` fed good cards to a "choose any number" prompt
The give-up ranking returned `2` for anything that was not a Strike or Defend,
so once the basics ran out it started offering real cards:
```
[023] give up Uppercut
[086] give up Stomp
[171] give up Bash <- the deck's only Vulnerable source
```
**Fix:** only basic Strikes and Defends are candidates. When none remain,
confirm and keep the good cards. A selection is forced only when
`can_confirm` is false.
## 20. `card_reward` had a redundant gate that skipped almost everything
A `want_any` Noul ("does this deck want any of these?") gated the whole
decision. When it was merely *uncertain* (0.54–0.59) the bot skipped, so it
skipped nearly every card reward and ran a 10-card deck.
**Fix:** the per-card Nouls ARE the signal. Skip only when no card clears
`CARD_PICK_THRESHOLD`.
## 21. A skipped card reward is NOT consumed — infinite loop
```
[409] skip_card_reward() # no card cleared 0.6
[410] claim_reward(index=2)
[411] skip_card_reward()
[412] claim_reward(index=2) ... forever
```
Skipping returns to the rewards screen with the card **still listed**. Claiming
it again reopens the card screen, and the cycle repeats.
Verified directly: `skip_card_reward` -> `ok`, then the rewards list still
contains `[2] card: Add a card to your deck.`
**Fix:** record that a card reward was skipped, and ignore card rewards on the
rewards screen afterwards. `rewards` and `card_reward` are declared one
**screen group** so the flag survives the hop between them — otherwise it is
cleared on every transition and the loop returns.
## 22. Module-level guards leaked across screens
Found by the new `test_brain.py`. A fresh fake-merchant shop was reported as
"unchanged after a purchase" because a shop signature from an earlier screen
was still set.
**Fix:** `_reset_screen_guards()` clears all per-screen state whenever the
screen group changes. Also made the shop guard precise: it now only fires when
we actually purchased from that exact shop state.
## 23. `fake_merchant` nests its inventory one level deeper
The shop is at `fake_merchant.shop.items`, not `shop.items`. Reading only
`obs["shop"]` made every fake-merchant shop look empty, so the bot left
immediately without buying.
**Fix:** resolve `obs["shop"]`, else `obs["fake_merchant"]["shop"]`, else
`obs["fake_merchant"]`.
## 24. `embark` is rejected without a character selected
```
action rejected: Embark button not available — select a character first
```
An earlier version assumed `embark` defaults to the first unlocked character.
It only worked once because a character happened to be selected already.
**Fix:** read the game's own `message`:
- `"Select a character."` -> select one
- `"Selected The Ironclad. Use 'confirm' to embark."` -> embark
## 25. `relic_select` used a diluted Choice and ignored `can_skip`
Verified shape: `relic_select.relics[]` with `index/id/name/description/rarity`
plus `can_skip`. It now uses the same re-ranking pattern, honours `can_skip`,
and falls back to the rarest relic rather than index 0.
## New: `test_brain.py`
A structural regression suite, deliberately separate from decision quality.
It asserts:
1. **Every `state_type` produces an action that is LEGAL for that state.**
Compares against `sts2.LEGAL_ACTIONS`. This is what would have caught a
handler emitting `end_turn` during someone else's turn.
2. **Every action the decision layer can emit is declared somewhere**, so a
typo cannot silently produce an invalid action.
3. **Every fallback respects its inputs** — removal does not target an
upgraded card, upgrade does not target a basic one, card reward takes the
rarest rather than index 0, `hand_select` gives up a Strike rather than
Bash.
54 assertions, runs offline with `client=None`, no model calls, no game.
Run it before every session.
---
# Session 4 — the toggle and transition traps
Found by running the bot for long stretches. Every one of these was a stall or a
crash, not a decision-quality issue.
## 26. `combat_select_card` TOGGLES — re-selecting deselects
The same trap as `card_select`, in a different action. Measured: **18
consecutive** `combat_select_card(card_index=0) # give up Defend` with the state
never changing, because selecting an already-selected card deselects it.
`hand_select.selected_cards` is the authoritative list of what is already
chosen. Exclude those from the candidate list.
**Watch out:** `cards[].index` and `selected_cards[].index` are **different
index spaces** (different arrays, independent counters). Names are the only
reliable way to match between them.
## 27. `hand_select` has a mode that only needs confirming
Measured state:
```
mode = "upgrade_select"
prompt = "Confirm Card to Upgrade"
cards = [(0, "Defend")]
selected = []
can_confirm = true
```
Sending `combat_select_card` is a silent no-op here. The card is already picked;
`combat_confirm_selection` closes the screen and moves to `monster`.
**Fix:** mode-aware.
| mode | action |
|---|---|
| `upgrade_select` | `combat_confirm_selection` |
| `simple_select` | select cards, then confirm when nothing selectable remains |
## 28. Character select: no indicator, and flaky embark
Two problems at once:
1. **The state carries no "selected" indicator.** The mod hardcodes
`result["message"] = "Select a character."` whatever is chosen. Verified in
`AddCharacterSelectMenuState`. So the message cannot be used to tell whether
a character is picked.
2. **Embarking right after selecting is flaky.** Measured three consecutive
`Embark button not available - select a character first` rejections, because
the selection had not registered yet.
**Fix:** ALTERNATE — select, embark, select, embark. A rejected embark is always
followed by a fresh select, so the sequence is self-correcting regardless of
timing.
An earlier attempt used signature comparison ("if the screen is unchanged, a
character was already selected"). It failed because the signature keeps changing
during the embark transition, restarting the cycle.
## 29. The run loop's guard blocked the brain's own recovery
`run.py` waited whenever the state was unchanged after a successful action, and
it did so **before calling `brain.decide`**. That made it impossible for any
handler to notice a repeated state and react differently — which is exactly how
`character_select` and multi-line dialogue work.
**Fix:** move the check to AFTER deciding, and suppress only a **repeated
action**, never acting in general. Proposing a *different* action is always
allowed.
## 30. A bounded retry is required
Even the corrected guard stalled on Ancient dialogue: one click was issued, the
state had not updated yet, and the guard then refused to retry forever.
**Fix:** `--max-duplicate-waits` (default 3). After that many suppressions on an
unchanged state, re-execute the action. Some actions legitimately need repeating
and some transitions are just slow.
## 31. `unknown` and `overlay` are transitions, not screens
Resetting the per-screen guards on them wiped state mid-flow. During embark the
state flickers through `unknown`, which cleared the character-select guard and
restarted the select/embark cycle.
**Fix:** `_reset_screen_guards` returns early for `unknown` and `overlay`.
## 32. `menu_screen` must be part of the screen key
`main`, `singleplayer`, `character_select` and `tutorial_prompt` all share
`state_type == "menu"` but have completely different valid actions. Keying only
on `state_type` meant moving from the main menu to character select did not
reset anything.
**Fix:** `_screen_group()` returns `menu:<menu_screen>` for the menu state.
## 33. `RemoteDisconnected` crashed a run
```
http.client.RemoteDisconnected: Remote end closed connection without response
```
`RemoteDisconnected` is an `http.client.HTTPException`, **not** a `URLError`, so
it escaped `jev.py`'s transport handler and killed the process mid-fight.
**Fix, both layers:**
- `jev.py` also catches `http.client.HTTPException` and `OSError`, and retries.
- `run.py` wraps `brain.decide` in a broad `except Exception`, falls back to the
deterministic handlers, and keeps playing.
## 34. Known, self-correcting: stale-state rejections
Occasionally the game rejects a card we chose:
```
Card 'Infection' cannot be played: HasUnplayableKeyword
Card 'Evil Eye' cannot be played: EnergyCostTooHigh
```
`can_play` comes from the game's own `card.CanPlay()`, so the state was correct
when read. A Jev call takes ~0.75 s, so the state we decided on can be about a
second stale by the time the action lands.
This is a **race, not a logic bug**, and it self-corrects: the rejection sets
`last_ok = False`, so the loop re-decides immediately against a fresh state
instead of waiting. Cost is one wasted action. No revalidation pass is needed.
## Test coverage after this session
`test_brain.py`: **78 assertions**. New this session:
- indices must come from the data, never from list position (hostile-index
cases where `index` deliberately differs from array position)
- `hand_select` skips already-chosen cards, including duplicate names
- `hand_select` confirms for `upgrade_select`
- character select alternates select/embark and resets correctly
- a `StubClient` makes the model paths testable offline and deterministically

174
docs/research/06-decision-architecture.md Normal file
View file

@ -0,0 +1,174 @@
# 06 — Decision architecture
## The core constraint that shapes everything
Jev is a classifier. It picks one option from a set **we** define. It cannot
plan, cannot invent actions, and cannot do arithmetic — and it reported 0.79
confidence on a wrong lethal check.
So the architecture is not "send state, get actions". It is:
```
game state ──▶ CODE computes facts and enumerates legal actions
──▶ JEV judges between the enumerated options
──▶ CODE gates on margin, executes exactly one action
──▶ re-observe
```
The arrow that matters is the first one. **The action space is generated by
code, never by the model.**
## Three layers
| Layer | Engine | Responsibility | Examples |
|---|---|---|---|
| Facts | pure Python | arithmetic, legality, thresholds | lethal, threat bucket, block deficit, deck counts |
| Tactics | **Jev** | preference between legal options | which card, which target, which node |
| Gate | code | when to trust Jev | margin thresholds, fallback heuristics |
### Why the split is exactly here
Jev is good at **semantic judgement over described options**. It is bad at
arithmetic, counting, multi-hop indirection, and large noisy states.
So the split puts every number in code, and every *preference* in Jev. The
model sees conclusions:
```json
{
"combat": {
"your_turn": true,
"energy": 3,
"your_health": "healthy",
"incoming_threat": "chip",
"lethal_available": false,
"enemies_you_can_kill_now": "none",
"enemies": [{"id": "NIBBIT_0", "name": "Nibbit", "hp_state": "healthy",
"incoming": 12, "intends": "This enemy intends to Attack for 12 damage."}],
"hand": [{"index": 2, "name": "Bash", "cost": 2, "type": "Attack",
"targets": "AnyEnemy", "text": "Deal 8 damage. Apply 2 Vulnerable."}]
}
}
```
Note `your_health: "healthy"` and `incoming_threat: "chip"` — buckets, not
numbers to compare. Note `lethal_available: false` — a conclusion computed by
`facts.py`.
## Decision precedence in combat
```
1. CODE: lethal proven by facts.py? -> execute the lethal line
2. CODE: no play phase? -> wait
3. JEV : which play is best? -> act if margin is sufficient
4. CODE: fallback heuristic -> act
```
Step 1 never consults the model. That is the direct consequence of the
arithmetic failure: `facts.py` proves lethal and a deterministic search
executes it. Across 759 loop steps, **33 lethal lines were executed by code**
and Jev was never asked "can I kill this".
Step 4 exists because a low-confidence answer must not become a guess. The
fallback is a documented heuristic adapted from the STS2MCP strategy notes.
## Current coverage
| Decision | Jev | Fallback | Status |
|---|---|---|---|
| Combat: card play + target | yes | heuristic | working |
| **Combat: potion use** | yes, plus a hard-need override | spend when lethal | working |
| Map pathing | yes | HP / gold heuristic | working |
| Card reward | yes (with `skip`) | take first | falls back often |
| Card select: upgrade/transform/remove | yes | select first | working |
| Event option | yes, stricter gate | keyword safety net | working |
| Relic select | yes | take first | shape unverified |
| **Shop purchase** | yes (re-ranking) | leave | working |
| **Treasure relic** | yes when >1 offered | take first | working |
| **Bundle select** | yes | first bundle | working |
| In-combat exhaust/discard select | **no** | Strikes, then Defends | heuristic |
| Rest site: heal vs upgrade | **no** | HP < 60% → heal | heuristic |
| Crystal sphere | **no** | skip | gap |
Combat now issues three kinds of action: `play_card`, `use_potion`, and
`end_turn`.
### Potion precedence, and why it is split
```
1. CODE: lethal by cards? -> play the lethal line
2. CODE: not play phase? -> wait
3. CODE: incoming hit lethal AND no card can prevent it
-> spend a potion (no model)
4. JEV : is a potion worth spending now? -> use it if confident
5. JEV : which play is best? -> play it
6. CODE: fallback heuristic -> act
```
Step 3 exists because Jev answered the soft potion question at **0.61**, just
under the 0.65 Noul floor, on a state where the next hit was lethal and no card
could prevent it. Delegating that decision would have lost the run. Code
decides **that** a potion must be spent; Jev decides **which**.
The gaps cluster in **resource spending** — potions, gold, and one-shot card
effects. That is the category that decides boss fights. Run 1 died to Vantom
holding all three potions, which is a direct consequence of the potion gap.
## Batching
All questions for one state go in **one** call. Questions are evaluated in
parallel, and measured latency barely moves with question count:
| Request | Time |
|---|---|
| 1 short question | 0.73 s |
| 3 questions, full combat state | 0.90 s |
So adding a potion question to the combat call costs **no extra latency**.
A question that is only sometimes relevant is still worth asking
(speculative fan-out); the code ignores answers it does not need.
Questions in one call are independent. If Q2 needs Q1's answer, use a second
call.
## The closed loop
```
observe -> decide -> execute ONE action -> observe again
```
Strictly closed, because **playing a card removes it from hand and shifts every
later index**. An action list computed up front would be wrong after the first
play. The same applies to reward lists, which re-index on every claim.
## Confidence gating
| Answer type | Gate |
|---|---|
| `Choice` | `top >= 0.45 and (top - runner_up) >= 0.20` |
| `Noul` | `abs(noul - 0.5) >= 0.15` |
| `Score` | `confidence >= threshold` |
Never gate a `Choice` on `confidence` alone. It measures peakedness and
therefore falls as the option count rises.
## Robustness rules
1. Never trust `status: ok`. Verify against the next state read.
2. Wait on transitions; do not stop.
3. Bound every wait, and bound consecutive rejections.
4. When the state repeats after a **successful** action, wait.
When it repeats after a **rejected** action, try something else.
5. Claim and select collections **right-to-left** when they re-index.
## Open questions
- Can a System One policy clear an act boss at all, or does the macro layer
need escalation to a reasoning model? Run 1 reached floor 16, so the plateau
is now measurable rather than assumed.
- Card reward fell back to "take the first" several times on thin margins.
Composite `Score` questions per axis, combined with weights in code, may
beat a single `Choice`.
- Deck tracking is currently a composition snapshot persisted to `deck.json`,
refreshed from combat states. It will drift if cards are removed outside
combat without an intervening fight.

206
docs/research/07-run-log.md Normal file
View file

@ -0,0 +1,206 @@
# 07 — Run log
Results per run. Source of truth is the game's own history file:
```
~/Library/Application Support/SlayTheSpire2/steam/<steamid>/modded/profile1/saves/history/<ts>.run
```
That file records `win`, `killed_by_encounter`, `seed`, `run_time`, `build_id`,
`ascension`, `was_abandoned`, and the full deck. **It is the evaluation
harness.** Read it rather than instrumenting the bot.
---
## Run 001 — Ironclad, standard, ascension 0
| Field | Value |
|---|---|
| Outcome | **Loss** |
| Killed by | `ENCOUNTER.VANTOM_BOSS` (Act 1 boss) |
| Floor reached | 16 |
| Seed | `JUS59Z32HF` |
| Run time | 2017 s (33 min) |
| Game build | `v0.107.1` |
| Abandoned | no |
### Progress
- Cleared 15 floors.
- Beat an elite: **Byrdonis**, 81 HP, 17-damage attack, `Territorial` (+1
Strength each turn).
- Fought Nibbits, Twig Slimes, Leaf Slimes, Shrinker Beetle, Inklings, Mawler,
Ruby Raiders (3-enemy fight), Vantom.
- Healed at rest sites at 43% and 24% HP.
### Final deck (19 cards, 1 upgraded)
`5× Strike` (one at upgrade level 1), `4× Defend`, `Bash`, `Setup Strike`,
`Inflame`, `Iron Wave`, `Rage`, `Stomp`, `Bludgeon`, `Battle Trance`,
`Second Wind`, `Bloodletting`.
### Relics (3)
`Burning Blood` (floor 1), `Gorget` (floor 10), `Vajra` (floor 11).
### Potions held at death — all three
```
POTION.EXPLOSIVE_AMPOULE slot 0
POTION.STRENGTH_POTION slot 1
POTION.ENERGY_POTION slot 2
```
**This is the headline finding.** The combat brain had no potion logic, so the
bot died holding an Explosive Ampoule, a Strength Potion, and an Energy Potion.
The STS2MCP strategy notes state the principle directly: *"Don't hoard potions.
Dying with full potions is the worst outcome."*
The run was lost on tactics, not on deck quality. The deck was reasonable for
floor 16 and the relic set was functional.
### Decision sources
| Source | Count |
|---|---|
| `code` | 141 |
| `jev` | 70 |
| `fallback` | 35 |
| Lethal lines executed by code | 33 |
Aggregate across 759 loop steps in 5 sessions.
### Notable Jev calls
| Situation | Jev chose | Confidence |
|---|---|---|
| Combat, 5 cards | Bash over Defend | 0.85 |
| Card reward | Inflame | 0.28 |
| Card reward | Rage | 0.40 |
| Card reward | Setup Strike | 0.33 |
| Map, 2 options | Shop | 0.97 |
| Map, 2 options | RestSite | 0.88 |
| Boss fight | Battle Trance, Bloodletting, Second Wind | 0.62–1.00 |
Several card-reward margins were thin enough that the gate fell back to "take
the first". Card reward is the weakest Jev decision in the current design.
### Bugs this run exposed
Eight infinite loops or stalls, all now fixed. See
[05-failure-modes.md](05-failure-modes.md). The most expensive were the
right-to-left reward indexing and the silently-dropped potion reward.
---
## Baseline to beat
| Metric | Run 001 |
|---|---|
| Win | no |
| Floor | 16 |
| Act 1 boss reached | yes |
| Act 1 boss killed | no |
Run 002 should be measured against this. The first question is not "does it
win" but "does it get past Vantom, and does it die holding potions again".
## Characters unlocked
| Character | Status |
|---|---|
| The Ironclad | unlocked |
| The Silent | unlocked (after the `NEOW_EPOCH` reveal) |
| The Regent | locked |
| The Necrobinder | locked |
| The Defect | locked |
`progress.save` recorded `pending_character_unlock: "CHARACTER.SILENT"` before
the reveal, then Silent became selectable.
## Seeded runs
Standard singleplayer exposes **no seed**, and `run` state carries no seed
field. The seed is visible only in the history file, after the run.
Seeded modes exist but were not available until the first epoch unlock:
| Mode | Seeded |
|---|---|
| `standard` | no |
| `daily` | yes (fixed per day) |
| `custom` | yes |
For a reproducible evaluation harness, `custom` is the likely path. This needs
verification — the mode screen appeared once and was not captured.
---
# Session 2 — potions, shops, treasure, events
## Scoreboard
| # | Win | Killed by | Seed | Time | Potions at end | Act reached |
|---|---|---|---|---|---|---|
| 001 | no | `VANTOM_BOSS` | `JUS59Z32HF` | 2017 s | **3** | 1 boss |
| 002 | no | `BYRDONIS_ELITE` | `MJ4J25A3BH` | 688 s | 0 | 1 |
| 003 | no | `SNAPPING_JAXFRUIT_NORMAL` | `HKLFARJJJQ` | 186 s | 0 | 1 |
| 004 | no | **`THE_INSATIABLE_BOSS`** | `MLQ4KBZQWV` | 2235 s | 2 | **2 boss** |
Run 004 reached the Act 2 boss with 11 relics and a 29-card deck. That is the
best result so far and the first time the bot cleared an act.
## Run 003 — the catastrophic event
Died to a **normal** encounter at 186 s. The cause was not combat:
```
[102] jev chose Keep Deciphering conf=0.28
[104] jev chose Lose Everything conf=0.49
```
The event offered "Keep Deciphering" and "Lose Everything". Jev picked
"Lose Everything", which **set the player's max HP to 1**. Every subsequent
state read then showed:
```
hp=1/1 (healthy)
```
The bot believed it was at full health, walked into a normal fight at 1 HP, and
died. Two separate defects, both now fixed:
1. `_hp_bucket` bucketed by percentage only, so `1/1` was 100% = "healthy".
Fixed: absolute HP <= 5 is always `critical`.
2. The event gate accepted a run-ending option at 0.49. Fixed with a stricter
event gate plus a deterministic keyword safety net. See
[05](05-failure-modes.md) and [02](02-system-one-jev.md).
## Run 004 — what the new decision loops did
| Decision | Count | Notes |
|---|---|---|
| `shop_purchase` | 6 in one shop | Lantern, Feel No Pain, Evil Eye, Salvo, Equilibrium, Headbutt |
| `use_potion` | 2+ | Fire Potion chosen by Jev at conf 1.00; two spent on lethal hits |
| `claim_treasure_relic` | 2 | Lucky Fysh, plus earlier Orichalcum and Bronze Scales |
| `combat_confirm_selection` | 4 | Hand-select now confirms instead of failing |
| event safety net | 1 | "uncertain (0.44); took safest option" |
Gold went from 528 to 130 across Act 2, so the "never buy anything" gap is
closed. Potions at death dropped from 3 (run 001) to 2 (run 004) while the run
went a full act further.
## Still open
- The bot still died holding 2 potions. The hard-need path only fires when the
incoming hit is *lethal*; it does not yet spend potions on a losing
attrition fight where HP is dropping every turn.
- Buying 6 items in one shop produced a 29-card deck. `SHOP_BUY_THRESHOLD`
may be too permissive; deck dilution is a real cost.
- Card reward still falls back to "take the first" fairly often.
## Recurring manual blocker
The Timeline epoch reveal blocked the menu again after run 004, this time with
`IRONCLAD2_EPOCH`. The mod refuses to automate it. A human must reveal it
before the next run can start. Expect this after most runs.

74
docs/research/README.md Normal file
View file

@ -0,0 +1,74 @@
# STS2 Bot — Research Notes
Living notes on reverse-engineering Slay the Spire 2 and driving it with a
TypeSafe System One (Jev) decision model.
These documents record **what was measured**, not what was assumed. Where a
claim comes from a live test, the evidence is quoted. Where something is a
guess, it says so.
## Contents
| # | Document | Covers |
|---|---|---|
| 01 | [Game engine and mod surface](01-game-engine-and-mod-surface.md) | Engine, assemblies, the official mod loader, manifest schema |
| 02 | [System One / Jev](02-system-one-jev.md) | What Jev is, measured latency and cost, the arithmetic failure |
| 03 | [STS2MCP interface](03-sts2mcp-interface.md) | The community mod, version drift, rebuilding from source |
| 04 | [State shapes](04-state-shapes.md) | Every verified JSON shape, per `state_type` |
| 05 | [Failure modes](05-failure-modes.md) | Every infinite loop found live, with its fix |
| 06 | [Decision architecture](06-decision-architecture.md) | The three-layer design and why the split is where it is |
| 07 | [Run log](07-run-log.md) | Results per run, with seeds and outcomes |
Architecture summary lives in [`../DESIGN.md`](../DESIGN.md).
## How to add a finding
1. Prefer a measurement over an inference. Quote the command and the output.
2. Put game facts in 01/03/04, model facts in 02, bugs in 05.
3. Record the **date** and the **game build** (`v0.107.1` today). Both move.
4. When a finding is later disproved, do not delete it. Mark it superseded
and say what replaced it. The wrong turn is often the useful part.
## Environment these notes were taken on
| Item | Value |
|---|---|
| Game build | `v0.107.1`, commit `59260271` |
| Platform | macOS (arm64), Steam |
| Engine | Godot 4.5.1 (.NET), runtime .NET 9.0.7 |
| Mod | STS2MCP, rebuilt from upstream `main` @ `55e0648` |
| Model | `jev-latest` resolving to `jev-1.13.0` |
## Rebuilding the mod
`STS2MCP` release `0.4.0` is **broken on this game build**. See
[03](03-sts2mcp-interface.md). To rebuild:
```bash
cd ~/sts2-bot/vendor/STS2MCP
nix shell nixpkgs#dotnet-sdk_9 --command bash -c '
dotnet build STS2_MCP.csproj -c Release -o out/STS2_MCP \
-p:STS2GameDir="$HOME/Library/Application Support/Steam/steamapps/common/Slay the Spire 2"'
cp out/STS2_MCP/STS2_MCP.dll \
"$HOME/Library/Application Support/Steam/steamapps/common/Slay the Spire 2/SlayTheSpire2.app/Contents/MacOS/mods/"
```
Then restart the game. Mods load only at process start.
## Testing
Two suites, both offline. Run them before every session.
```bash
python3 test_brain.py # 54 assertions — structural / programmatic
python3 test_facts.py # 29 assertions — arithmetic and parsing
```
`test_brain.py` is the important one for catching usage bugs. It asserts that
every `state_type` produces an action **legal for that state**, that every
action the decision layer can emit is declared somewhere, and that every
fallback respects its own inputs. It needs no model and no running game.
It was added after a session in which four separate infinite loops and three
hardcoded fallbacks were found by hand. Most of them would have been caught
here.