Add design doc and research notes
DESIGN.md covers the three-layer architecture (facts in code, Jev for tactics, gated escalation for macro). research/ documents the engine and mod surface, the Jev classifier's measured behavior, the STS2MCP HTTP interface, state shapes, failure modes, decision architecture, and a run log of the first four sessions.
This commit is contained in:
0xrsydn
commit
fb32822468
9 changed files with 2341 additions and 0 deletions
174
docs/research/01-game-engine-and-mod-surface.md
Normal file
174
docs/research/01-game-engine-and-mod-surface.md
Normal file
|
|
@ -0,0 +1,174 @@
|
|||
# 01 — Game engine and mod surface
|
||||
|
||||
Measured on game build `v0.107.1` (commit `59260271`), macOS arm64, Steam.
|
||||
|
||||
## Engine
|
||||
|
||||
Slay the Spire 2 is a **Godot 4.5.1 (.NET)** game, not Java like the original.
|
||||
This matters: the gameplay code is plain managed C# IL, so it can be read
|
||||
directly with no decompiler.
|
||||
|
||||
| Item | Value |
|
||||
|---|---|
|
||||
| Engine | Godot 4.5.1 (.NET build) |
|
||||
| Runtime | .NET 9.0.7, self-contained `osx-arm64` |
|
||||
| Root namespace | `MegaCrit.Sts2.*` (2,751 types) |
|
||||
| Bundle id | `com.megacrit.SlayTheSpire2` |
|
||||
|
||||
## Layout inside the app bundle
|
||||
|
||||
```
|
||||
SlayTheSpire2.app/Contents/
|
||||
MacOS/
|
||||
Slay the Spire 2 179 MB Mach-O universal (x86_64 + arm64)
|
||||
mods/ <- mods load from here
|
||||
Frameworks/
|
||||
libfmod.dylib, libfmodstudio.dylib
|
||||
libGodotFmod.macos.template_release.framework
|
||||
libspine_godot.macos.template_release.framework
|
||||
libSentry.dylib
|
||||
Resources/
|
||||
Slay the Spire 2.pck 1.9 GB Godot pack
|
||||
data_sts2_macos_arm64/ .NET assemblies + runtime
|
||||
sts2.dll 9.3 MB main game assembly
|
||||
sts2.xml 5.3 MB XML docs, 19,635 members
|
||||
GodotSharp.dll, 0Harmony.dll, MonoMod.Backports.dll,
|
||||
Steamworks.NET.dll, Sentry.dll, ...
|
||||
data_sts2_macos_x86_64/ same, other arch
|
||||
```
|
||||
|
||||
### The free win: `sts2.xml`
|
||||
|
||||
`sts2.xml` is the .NET XML documentation file, shipped un-stripped. It gives
|
||||
**19,635 documented members** with prose summaries. Example, verbatim:
|
||||
|
||||
> `ModInitializerAttribute` — *"Declares a class as the main entry point for
|
||||
> the mod. If this is present, then upon loading the mod, we'll call the method
|
||||
> named `initializerMethod` within the class. Otherwise, we'll create a harmony
|
||||
> instance for the mod and call `Harmony.PatchAll`."*
|
||||
|
||||
This removes most of the guesswork from modding. Query it with
|
||||
`~/sts2-re/docdump.py` or any XML parser.
|
||||
|
||||
### Harmony and MonoMod ship in the game
|
||||
|
||||
`0Harmony.dll` and `MonoMod.*.dll` are present in the shipping build. That is
|
||||
a strong signal that runtime patching is a supported path, not an accident.
|
||||
|
||||
## Official mod loader
|
||||
|
||||
The game has first-class mod support. Namespace `MegaCrit.Sts2.Core.Modding`.
|
||||
|
||||
| Type | Purpose |
|
||||
|---|---|
|
||||
| `ModManager` | Discovers and loads mods |
|
||||
| `ModInitializerAttribute` | Declares a mod entry point |
|
||||
| `ModManifest` | The JSON manifest |
|
||||
| `ModHelper` | Content registration and hook subscription |
|
||||
| `ModSource` | `ModsDirectory` or `SteamWorkshop` |
|
||||
| `SettingsSaveMod` | Per-mod enable/disable and manual load order |
|
||||
|
||||
`ModManager.Initialize` is documented as loading mods *"from the 'mods'
|
||||
directory next to the executable, as well as Steam workshop files"* and is
|
||||
*"called as early as possible in the game's initialization process"*.
|
||||
|
||||
**Consequence:** mods load at process start. Changing a mod requires a full
|
||||
restart. "Save and quit" to the main menu is not enough.
|
||||
|
||||
### Mod directories
|
||||
|
||||
| Source | Path |
|
||||
|---|---|
|
||||
| Local (macOS) | `SlayTheSpire2.app/Contents/MacOS/mods/` |
|
||||
| Steam Workshop | `Steam/steamapps/workshop/content/2868840/` |
|
||||
|
||||
### Manifest schema (verified from a working mod)
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "STS2_MCP",
|
||||
"name": "STS2 MCP",
|
||||
"author": "kunology",
|
||||
"description": "MCP server bridge for Slay the Spire 2",
|
||||
"version": "0.4.0",
|
||||
"has_pck": false,
|
||||
"has_dll": true,
|
||||
"affects_gameplay": false
|
||||
}
|
||||
```
|
||||
|
||||
Note **snake_case** in JSON (`affects_gameplay`, `has_dll`, `has_pck`), which
|
||||
differs from the C# property names (`affectsGameplay`). The manifest filename
|
||||
must match the mod id: `STS2_MCP.json` for id `STS2_MCP`.
|
||||
|
||||
### Mod loading, from the live log
|
||||
|
||||
```
|
||||
Found mod manifest file .../mods/STS2_MCP.json
|
||||
Mods have been re-sorted because we detected a change or dependency order was broken.
|
||||
[WARN] Mod STS2_MCP does not declare min game version. Assuming that it is supported.
|
||||
Loading assembly DLL .../mods/STS2_MCP.dll
|
||||
Calling initializer method of type STS2_MCP.McpMod for STS2_MCP, Version=1.0.0.0
|
||||
Finished mod initialization for 'STS2 MCP' (STS2_MCP).
|
||||
--- RUNNING MODDED! --- Loaded 1 mods (1 total)
|
||||
[Sentry.NET] Is running modded
|
||||
```
|
||||
|
||||
`minGameVersion` is optional but its absence produces a warning. It is worth
|
||||
declaring in any mod we write.
|
||||
|
||||
### Consent flow
|
||||
|
||||
`NConfirmModLoadingPopup` is documented as *"Vertical popup used to get player
|
||||
confirmation before loading mods. Renders above the capstone screens (above top
|
||||
bar)."* It appears **over the main menu, at launch**, and only when mods are
|
||||
present. Consent persists as `mod_settings: {"mods_enabled": true}` in
|
||||
`settings.save`.
|
||||
|
||||
There is also a `nomods` command-line argument that makes `ModManager` skip
|
||||
initialization entirely (`ModManagerState.Skipped`).
|
||||
|
||||
## Save layout
|
||||
|
||||
Modded and vanilla progress are **isolated**:
|
||||
|
||||
```
|
||||
~/Library/Application Support/SlayTheSpire2/steam/<steamid>/
|
||||
settings.save {"mod_settings": {"mods_enabled": true}, ...}
|
||||
profile.save
|
||||
profile1/saves/ vanilla progress
|
||||
modded/profile1/saves/ MODDED progress
|
||||
prefs.save
|
||||
progress.save unlocks, card/enemy/encounter stats
|
||||
current_run.save the run in progress
|
||||
history/<ts>.run completed run records
|
||||
```
|
||||
|
||||
Modded runs never touch vanilla saves. This was confirmed by the log line
|
||||
`Profile-scoped data path initialized: user://steam/<id>/modded/profile1`.
|
||||
|
||||
## Bundle signing
|
||||
|
||||
The app is **ad-hoc signed** (`Signature=adhoc`, `flags=0x10002(adhoc,runtime)`)
|
||||
and `spctl -a` already returns `rejected`. There is no quarantine attribute
|
||||
(only `com.apple.provenance`). Adding files to `Contents/MacOS/` breaks the
|
||||
sealed-resource count but there is no enforced signature, and Steam launches
|
||||
the binary directly.
|
||||
|
||||
**Do not run Steam's "Verify integrity of game files"** — it deletes the mods
|
||||
directory, because those files are not in the depot manifest.
|
||||
|
||||
## Useful namespaces for a bot
|
||||
|
||||
| Namespace | Why it matters |
|
||||
|---|---|
|
||||
| `Core.Modding` | Official mod API |
|
||||
| `Core.AutoSlay` | MegaCrit's own auto-play bot, shipped in the build |
|
||||
| `Core.DevConsole.ConsoleCommands` | ~40 built-in debug commands |
|
||||
| `Core.GameActions` | Command pattern for game actions |
|
||||
| `Core.Runs` / `Core.Combat` | Run and combat state |
|
||||
| `Core.Saves` | Source-generated JSON serialization |
|
||||
|
||||
`AutoSlayer.Start(seed, ...)` runs a complete automated game with handlers for
|
||||
every screen and room. It is MegaCrit's smoke-test harness and is the best
|
||||
available reference for driving the game programmatically.
|
||||
199
docs/research/02-system-one-jev.md
Normal file
199
docs/research/02-system-one-jev.md
Normal file
|
|
@ -0,0 +1,199 @@
|
|||
# 02 — System One / Jev
|
||||
|
||||
Everything here is measured against `jev-latest` → `jev-1.13.0` from
|
||||
`https://api.typesafe.ai/v1/systemone`.
|
||||
|
||||
## What Jev is
|
||||
|
||||
TypeSafe states it plainly:
|
||||
|
||||
> "System One is TypeSafe's model for building AI-powered software, not agents.
|
||||
> It does not generate code or choose its own next action."
|
||||
|
||||
It is a **calibrated classifier**. It evaluates a `state` and answers typed
|
||||
questions about it. It cannot plan, cannot call tools, and cannot invent values
|
||||
outside the option set you supply.
|
||||
|
||||
| Primitive | Shape | Returns |
|
||||
|---|---|---|
|
||||
| `Choice` | pick 1 of ≤ 255 options | `choice`, `probabilities`, `confidence` |
|
||||
| `Score` | position on ordered levels | `score`, `legend`, `probabilities`, `confidence` |
|
||||
| `Noul` | yes/no | `noul` (probability only, **no confidence**) |
|
||||
|
||||
Documented properties: questions in one call are evaluated **independently and
|
||||
in parallel**. One answer is never hidden context for another.
|
||||
|
||||
## Measured latency — 7× the documented figure
|
||||
|
||||
The docs say "most queries complete in about 100 ms". Measured end-to-end from
|
||||
this machine:
|
||||
|
||||
| Request | Time |
|
||||
|---|---|
|
||||
| Minimal (1 short noul) | 0.73 s |
|
||||
| Minimal × 5 repeats | 0.73, 0.73, 0.74, 0.80, 0.76 s |
|
||||
| Full combat state, 3 questions | 0.90 s |
|
||||
| `GET /v1/models` TTFB | 0.68 s (connect 0.22 s) |
|
||||
|
||||
The floor is **~0.73 s**, dominated by network RTT plus server TTFB.
|
||||
|
||||
**The useful consequence:** a 1-question call and a 3-question call over a full
|
||||
combat state differ by only 0.17 s. State size and question count are nearly
|
||||
free. **Batch every question for a state into one call.**
|
||||
|
||||
## Cost
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Input | $0.042 per Mtok |
|
||||
| Output | free |
|
||||
| Rate limit | 250k tok/s, 1,200 req/min |
|
||||
| Context | 64k total; 32k for `state` + longest question |
|
||||
| Input type | text only (string, JSON object, array) |
|
||||
|
||||
For comparison, the STS2MCP README reports a full run costs a frontier LLM
|
||||
about **8M tokens** (~$20–40). The same volume through Jev is roughly
|
||||
**$0.08–0.35 per run**.
|
||||
|
||||
## THE CRITICAL FINDING: Jev cannot do arithmetic
|
||||
|
||||
Ground truth: `energy 3`, hand `Strike(1 cost, 6 dmg) ×3` plus
|
||||
`Bash(2 cost, 8 dmg)`, target on `19 HP`. Maximum reachable damage is **18**,
|
||||
so lethal is **NO**.
|
||||
|
||||
| Question | Jev answered | Verdict |
|
||||
|---|---|---|
|
||||
| Max damage bucket | `18_to_23` @ 0.73 | **Correct** |
|
||||
| Is lethal available? | `0.79` | **Wrong** |
|
||||
|
||||
Jev bucketed the magnitude correctly but failed the threshold comparison of 18
|
||||
versus 19 — **and reported 0.79 confidence on the wrong answer**.
|
||||
|
||||
Two conclusions:
|
||||
|
||||
1. Jev is decent at *approximate magnitude*, poor at *exact comparison*.
|
||||
2. **Confidence gating cannot protect against arithmetic errors.** A
|
||||
0.79-confidence wrong answer passes any sane threshold.
|
||||
|
||||
This is why `facts.py` computes every sum, comparison, and threshold, and hands
|
||||
Jev only conclusions (`"lethal_available": true`). This is a regression test:
|
||||
see `test_facts.py` case 1.
|
||||
|
||||
The jaggedness page confirms the general shape: Jev "does not count reliably",
|
||||
"is not a calculator", degrades with indirection, and suffers "context rot"
|
||||
when the state carries irrelevant detail.
|
||||
|
||||
## The confidence gate trap
|
||||
|
||||
**A fixed confidence floor is wrong when the option count varies.**
|
||||
|
||||
Measured: 5 cards offered. Jev picks `Bash` at **0.61**, with `Defend` at 0.29.
|
||||
Reported `confidence` is **0.50**.
|
||||
|
||||
Why: confidence measures *peakedness*. For 5 options the documented formula is
|
||||
`(count × peak − 1) / (count − 1)` = `(5 × 0.61 − 1) / 4` = 0.50.
|
||||
|
||||
A 0.55 floor rejected a clear plurality and fell back to a worse heuristic.
|
||||
With 5 options, 0.50 is a strong plurality; with 2 options, 0.50 is a coin flip.
|
||||
|
||||
**Fix:** gate a `Choice` on margin over the runner-up, which is scale-free:
|
||||
|
||||
```python
|
||||
top = probabilities[choice]
|
||||
runner = max(v for k, v in probabilities.items() if k != choice)
|
||||
act = top >= 0.45 and (top - runner) >= 0.20
|
||||
```
|
||||
|
||||
`Noul` has no confidence field, so gate it on distance from 0.5:
|
||||
|
||||
```python
|
||||
act = abs(noul - 0.5) >= 0.15 # act when noul >= 0.65 or <= 0.35
|
||||
```
|
||||
|
||||
## Design rules that follow from the above
|
||||
|
||||
1. **Never** route an arithmetic comparison through Jev. Compute it in code.
|
||||
2. Send **conclusions and buckets**, not raw numbers to be compared.
|
||||
3. Keep `state` small and relevant — context rot is real and measurable.
|
||||
4. Batch all questions for one state into **one** call.
|
||||
5. Questions in one call are independent; if Q2 needs Q1's answer, make a
|
||||
second call.
|
||||
6. Gate `Choice` on margin, not confidence.
|
||||
7. Treat a low-confidence answer as a reason to fall back, not to guess.
|
||||
|
||||
## Credential handling
|
||||
|
||||
The key lives in a sops-nix managed file, **not** in the shell environment:
|
||||
|
||||
```
|
||||
~/.config/secrets/global-env/TYPESAFEAI_API_KEY
|
||||
-> ~/.config/sops-nix/secrets/TYPESAFEAI_API_KEY
|
||||
```
|
||||
|
||||
`jev.py` reads it at runtime and never logs it. `JevClient.__repr__` prints
|
||||
`key=REDACTED`, so the secret cannot leak through a traceback or log line.
|
||||
|
||||
---
|
||||
|
||||
# Session 2 additions
|
||||
|
||||
## Option count dilutes a Choice — measured twice
|
||||
|
||||
### Shop, 14 candidates
|
||||
|
||||
A shop offered 14 affordable items in one `Choice`. Jev's top pick scored only
|
||||
**0.26** (runner 0.17, margin 0.09) — the probability mass spread across all
|
||||
fourteen. The margin gate correctly rejected it, so the bot would always leave
|
||||
the shop with gold unspent.
|
||||
|
||||
**Fix:** use the documented **re-ranking** pattern — one *absolute* `Noul` per
|
||||
candidate, then take the argmax in code. Absolute judgements do not dilute as
|
||||
the candidate count grows.
|
||||
|
||||
### Same problem, smaller: 5 cards
|
||||
|
||||
Already covered above. With 5 options, `confidence` is `(5 × top − 1) / 4`, so
|
||||
a clear plurality reads as 0.50. Gate on margin, not confidence.
|
||||
|
||||
## The question framing matters more than the threshold
|
||||
|
||||
Asking Jev to weigh value against a **number** degrades its judgement, exactly
|
||||
as the jaggedness page predicts. Measured on the same shop, same state:
|
||||
|
||||
| Framing | Spread across candidates | Top item |
|
||||
|---|---|---|
|
||||
| "worth its **72 gold** price for this deck?" | **0.28** | Bag of Preparation 0.51 |
|
||||
| "would this make this deck stronger?" | **0.48** | Bag of Preparation 0.67 |
|
||||
| "does this fit what this deck is doing?" | 0.51 | Ashen Strike 0.71 |
|
||||
| "improve more than it dilutes?" | 0.42 | Bag of Preparation 0.65 |
|
||||
|
||||
Putting the price in the question **halved the spread** and pulled the top item
|
||||
below any usable threshold.
|
||||
|
||||
**Rule:** filter affordability in code, keep the price in the `state` for
|
||||
context, and keep it **out of the question**. Ask about deck fit only.
|
||||
|
||||
## Risk is not a preference, and Jev is bad at spotting danger
|
||||
|
||||
On an event offering "Keep Deciphering" and "Lose Everything":
|
||||
|
||||
| Option | "Does this risk losing the run?" |
|
||||
|---|---|
|
||||
| "Lose Everything" | **0.46** |
|
||||
| "Keep Deciphering" | 0.52 |
|
||||
| "Stop" | 0.38 |
|
||||
|
||||
The model ranked the run-ending option as less risky than a moderate one. Do
|
||||
not use a model judgement to detect danger. Use deterministic keyword matching
|
||||
plus a stricter confidence gate. See [05](05-failure-modes.md) §10.
|
||||
|
||||
## Confirmed working patterns
|
||||
|
||||
| Pattern | Where used | Result |
|
||||
|---|---|---|
|
||||
| Batch all questions for one state in one call | combat | 0.73 s for 1 question, 0.90 s for 3 |
|
||||
| Absolute `Noul` per candidate, argmax in code | shop | Works where a 14-way Choice failed |
|
||||
| Compute the hard fact in code, let Jev pick | lethal, potions | 33 lethal lines executed without the model |
|
||||
| Gate on margin, not confidence | every `Choice` | Fixed a rejected-correct-answer bug |
|
||||
| Keep arithmetic out of the question | shop | Doubled the usable spread |
|
||||
| Deterministic safety net for danger | events | Caught what the model missed |
|
||||
168
docs/research/03-sts2mcp-interface.md
Normal file
168
docs/research/03-sts2mcp-interface.md
Normal file
|
|
@ -0,0 +1,168 @@
|
|||
# 03 — STS2MCP interface
|
||||
|
||||
A community mod by `kunology` that exposes the running game over a local HTTP
|
||||
API. Repository: `github.com/Gennadiyev/STS2MCP`, MIT.
|
||||
|
||||
It is the actuator for this project: it reads state and performs one action at
|
||||
a time. It does **not** alter gameplay (`affects_gameplay: false`).
|
||||
|
||||
## Endpoints
|
||||
|
||||
Base URL `http://localhost:15526`, no authentication.
|
||||
|
||||
| Method | Path | Purpose |
|
||||
|---|---|---|
|
||||
| GET | `/api/v1/singleplayer?format=json\|markdown` | Read state |
|
||||
| POST | `/api/v1/singleplayer` | Perform one action |
|
||||
| GET | `/api/v1/multiplayer` | Read multiplayer state |
|
||||
| POST | `/api/v1/multiplayer` | Multiplayer action |
|
||||
| GET | `/api/v1/profile` | Persistent profile progress |
|
||||
| GET | `/api/v1/compendium` | Compendium-shaped progress |
|
||||
| GET | `/api/v1/wiki?query=&item_type=&limit=` | Fuzzy card/relic lookup |
|
||||
| GET/POST | `/api/v1/profiles` | List / switch / delete profile slots |
|
||||
|
||||
All POST bodies carry an `"action"` field. Responses carry
|
||||
`{"status": "ok"|"error", "message": ...}`.
|
||||
|
||||
## `state_type` is the action enumerator
|
||||
|
||||
Every response carries `state_type`, and each type has a small fixed action
|
||||
set. **This is the action space.** There is no need to write one.
|
||||
|
||||
| `state_type` | Legal actions |
|
||||
|---|---|
|
||||
| `menu`, `game_over` | `menu_select` |
|
||||
| `monster` / `elite` / `boss` | `play_card`, `use_potion`, `end_turn` |
|
||||
| `hand_select` | `combat_select_card`, `combat_confirm_selection` |
|
||||
| `rewards` | `claim_reward`, `proceed` |
|
||||
| `card_reward` | `select_card_reward`, `skip_card_reward` |
|
||||
| `map` | `choose_map_node` |
|
||||
| `event` | `choose_event_option`, `advance_dialogue` |
|
||||
| `rest_site` | `choose_rest_option`, `proceed` |
|
||||
| `shop`, `fake_merchant` | `shop_purchase`, `proceed` |
|
||||
| `treasure` | `claim_treasure_relic`, `proceed` |
|
||||
| `card_select` | `select_card`, `confirm_selection`, `cancel_selection` |
|
||||
| `bundle_select` | `select_bundle`, `confirm_bundle_selection`, `cancel_bundle_selection` |
|
||||
| `relic_select` | `select_relic`, `skip_relic_selection` |
|
||||
| `crystal_sphere` | `crystal_sphere_set_tool`, `_click_cell`, `_proceed` |
|
||||
| `overlay`, `unknown` | none (transitions, or manual) |
|
||||
|
||||
`use_potion` and `discard_potion` work in any state where potions are
|
||||
reachable, including outside combat.
|
||||
|
||||
## VERSION DRIFT — release 0.4.0 is broken on this build
|
||||
|
||||
### Symptom
|
||||
|
||||
`GET /api/v1/singleplayer` returns **HTTP 500** as soon as combat starts:
|
||||
|
||||
```json
|
||||
{
|
||||
"error": "Failed to read game state: Method not found:
|
||||
'Boolean MegaCrit.Sts2.Core.Combat.CombatManager.get_IsPlayPhase()'.",
|
||||
"exception_type": "System.MissingMethodException",
|
||||
"stack_trace": " at STS2_MCP.McpMod.BuildBattleState(RunState, CombatRoom)"
|
||||
}
|
||||
```
|
||||
|
||||
Menu, character select, map, rewards, and events all work. **Only combat
|
||||
fails** — the one state that matters most.
|
||||
|
||||
### Root cause
|
||||
|
||||
`CombatManager.IsPlayPhase` was removed in game `v0.107.1`. Verified:
|
||||
|
||||
```bash
|
||||
grep -c IsPlayPhase sts2.xml # -> 0
|
||||
```
|
||||
|
||||
Replacements present in `v0.107.1`:
|
||||
|
||||
- `MegaCrit.Sts2.Core.Combat.PlayerTurnPhase.Play` (per-player)
|
||||
- `PlayerCombatState.Phase`
|
||||
- `ActionSynchronizerCombatState.PlayPhase` / `.NotPlayPhase`
|
||||
- `CombatManager.DebugOnlyGetState().CurrentSide == CombatSide.Player`
|
||||
|
||||
### Timeline
|
||||
|
||||
| Event | Date |
|
||||
|---|---|
|
||||
| Release `0.4.0` (broken) | 2026-05-05 |
|
||||
| Fix commit `55e06485`, PR #123 | 2026-07-29 |
|
||||
| Newer tag | none |
|
||||
| CI build artifact | none |
|
||||
| Fork shipping a prebuilt DLL | none (checked 12 forks) |
|
||||
|
||||
The fix exists in source but was never released.
|
||||
|
||||
### Resolution
|
||||
|
||||
Build from upstream `main`. The fix is present and documented in the code:
|
||||
|
||||
```csharp
|
||||
// STS2 v0.107 removed CombatManager.IsPlayPhase. The turn phase now lives per-player on
|
||||
// PlayerCombatState.Phase
|
||||
internal static bool IsPlayPhase(Player? player)
|
||||
{
|
||||
if (!IsPlayerSideTurn()) return false;
|
||||
return player?.PlayerCombatState?.Phase == PlayerTurnPhase.Play;
|
||||
}
|
||||
```
|
||||
|
||||
```bash
|
||||
nix shell nixpkgs#dotnet-sdk_9 --command bash -c '
|
||||
dotnet build STS2_MCP.csproj -c Release -o out/STS2_MCP \
|
||||
-p:STS2GameDir="$HOME/Library/Application Support/Steam/steamapps/common/Slay the Spire 2"'
|
||||
```
|
||||
|
||||
`Build succeeded. 0 Warning(s) 0 Error(s).` Result: 236,544 B DLL, replacing
|
||||
the 194,560 B release. The release DLL is kept at
|
||||
`vendor/STS2_MCP.dll.release-0.4.0`.
|
||||
|
||||
**Lesson:** the game moves faster than the mod's releases. Upstream open issues
|
||||
#131 and #132 already track `v0.111` compatibility. Expect this class of break
|
||||
to recur after any game update. Pin the mod build to the game build.
|
||||
|
||||
## Known doc drift inside the mod
|
||||
|
||||
`AGENTS.md` in the repo uses MCP **tool** names; `docs/raw-simplified.md` uses
|
||||
the HTTP **action** names. They do not match:
|
||||
|
||||
| AGENTS.md | HTTP action |
|
||||
|---|---|
|
||||
| `combat_end_turn` | `end_turn` |
|
||||
| `event_choose_option` | `choose_event_option` |
|
||||
| `rewards_pick_card` | `select_card_reward` |
|
||||
| `rest_choose_option` | `choose_rest_option` |
|
||||
| `proceed_to_map` | `proceed` |
|
||||
|
||||
Trust `raw-simplified.md` and the live API for direct HTTP use.
|
||||
|
||||
## Hard blocker: Timeline cannot be automated
|
||||
|
||||
After a run ends with a pending epoch, the mod refuses:
|
||||
|
||||
```json
|
||||
{"status":"error",
|
||||
"error":"Timeline has obtained epochs that still need to be revealed manually;
|
||||
not opening Timeline because this game state logs invalid unlock-state
|
||||
errors when entered through automation",
|
||||
"pending_epoch_ids":["NEOW_EPOCH"],
|
||||
"manual_action_required":true}
|
||||
```
|
||||
|
||||
This is a **deliberate guard**, not a bug. Entering Timeline through automation
|
||||
would corrupt unlock state. A human must open the Timeline and reveal the
|
||||
epoch. Until then the main menu offers only `settings` and `quit` and no run
|
||||
can be started.
|
||||
|
||||
## Mod config
|
||||
|
||||
The mod writes `mods/STS2_MCP.conf`:
|
||||
|
||||
```json
|
||||
{ "port": 15526 }
|
||||
```
|
||||
|
||||
It also injects an **"Instant Mode"** checkbox into the game settings, which
|
||||
shortens animations. Worth enabling for batch runs.
|
||||
430
docs/research/04-state-shapes.md
Normal file
430
docs/research/04-state-shapes.md
Normal file
|
|
@ -0,0 +1,430 @@
|
|||
# 04 — State shapes
|
||||
|
||||
Every shape below was captured from a **live game**, not read from docs. Keys
|
||||
are exact. Where a field is missing in some states, that is called out, because
|
||||
each absence caused a real bug.
|
||||
|
||||
Captured from `GET /api/v1/singleplayer?format=json` on game `v0.107.1`.
|
||||
|
||||
---
|
||||
|
||||
## `menu`
|
||||
|
||||
```json
|
||||
{
|
||||
"state_type": "menu",
|
||||
"menu_screen": "main",
|
||||
"message": "Main menu.",
|
||||
"options": ["singleplayer", "multiplayer", "compendium", "timeline", "settings", "quit"],
|
||||
"blocked_options": [
|
||||
{"name": "timeline", "enabled": false,
|
||||
"reason": "manual_epoch_reveal_required",
|
||||
"pending_epoch_ids": ["NEOW_EPOCH"]}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
`options` entries are **plain strings** here. On other screens they are objects
|
||||
`{"name": ..., "enabled": ...}`. Handle both.
|
||||
|
||||
Observed `menu_screen` values: `main`, `character_select`, `tutorial_prompt`.
|
||||
|
||||
Mode selection (after the first epoch unlock) offers
|
||||
`standard`, `daily`, `custom`, `back`. `daily` and `custom` are the seeded
|
||||
modes; standard singleplayer exposes no seed.
|
||||
|
||||
## `character_select`
|
||||
|
||||
```json
|
||||
{
|
||||
"state_type": "menu",
|
||||
"menu_screen": "character_select",
|
||||
"message": "Select a character.",
|
||||
"characters": [{
|
||||
"name": "The Ironclad", "id": "IRONCLAD", "locked": false,
|
||||
"hp": 80, "gold": 99, "energy": 3,
|
||||
"description": "Ironclad cards will now appear in rewards and shops.",
|
||||
"starting_relics": [{"name": "Burning Blood", "description": "At the end of combat, heal 6 HP."}],
|
||||
"starting_deck": ["Strike","Strike","Strike","Strike","Strike",
|
||||
"Defend","Defend","Defend","Defend","Bash"],
|
||||
"total_cards": 87, "total_relics": 8, "total_potions": 3
|
||||
}],
|
||||
"options": [{"name": "IRONCLAD", "enabled": true}, {"name": "SILENT", "enabled": false},
|
||||
{"name": "confirm", "enabled": true}, {"name": "embark", "enabled": true},
|
||||
{"name": "back", "enabled": true}]
|
||||
}
|
||||
```
|
||||
|
||||
All five characters are listed, with `locked` set per profile. `starting_deck`
|
||||
is the cheapest reliable source of the initial deck for a deck tracker.
|
||||
|
||||
## `monster` / `elite` / `boss`
|
||||
|
||||
```json
|
||||
{
|
||||
"state_type": "monster",
|
||||
"battle": {
|
||||
"round": 1, "turn": "player", "is_play_phase": true,
|
||||
"enemies": [{
|
||||
"entity_id": "NIBBIT_0", "combat_id": 1, "name": "Nibbit",
|
||||
"hp": 44, "max_hp": 44, "block": 0,
|
||||
"status": [{"id": "TERRITORIAL_POWER", "name": "Territorial", "amount": 1,
|
||||
"type": "Buff", "description": "At the end of Byrdonis's turn, it gains 1 Strength.",
|
||||
"keywords": [{"name": "Strength", "description": "..."}]}],
|
||||
"intents": [{"type": "Attack", "label": "12", "title": "Aggressive",
|
||||
"description": "This enemy intends to Attack for 12 damage."}]
|
||||
}]
|
||||
},
|
||||
"run": {"act": 1, "floor": 1, "ascension": 0},
|
||||
"player": {
|
||||
"character": "The Ironclad",
|
||||
"hp": 80, "max_hp": 80, "block": 0,
|
||||
"energy": 3, "max_energy": 3,
|
||||
"hand": [{
|
||||
"id": "STRIKE_IRONCLAD", "name": "Strike", "type": "Attack",
|
||||
"cost": "1", "star_cost": null,
|
||||
"description": "Deal 6 damage.", "rarity": "Basic", "is_upgraded": false,
|
||||
"keywords": [], "index": 1, "target_type": "AnyEnemy",
|
||||
"can_play": true, "unplayable_reason": null
|
||||
}],
|
||||
"draw_pile_count": 5, "discard_pile_count": 0, "exhaust_pile_count": 0,
|
||||
"draw_pile": [{"name": "Defend", "cost": "1", "star_cost": null, "description": "Gain 5 Block."}],
|
||||
"discard_pile": [], "exhaust_pile": [],
|
||||
"gold": 99, "status": [], "relics": [], "potions": [], "max_potion_slots": 3
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Critical details:
|
||||
|
||||
- **`cost` is a STRING** (`"1"`), not an int. Parse it. Some cards may be `"X"`.
|
||||
- **`intents[].label` carries the damage as a STRING** (`"12"`), and may be
|
||||
`"12x2"` for multi-hit. This is the authoritative number for incoming damage.
|
||||
- **Pile cards carry only `{name, cost, star_cost, description}`** — no `id`,
|
||||
no `type`, no `index`. Deck composition must therefore be counted **by name**,
|
||||
not by type. This caused a bug.
|
||||
- `intents` is absent when the enemy has no next move.
|
||||
- `status` entries are powers: `{id, name, amount, type, description, keywords}`.
|
||||
- `target_type` values seen: `Self`, `AnyEnemy`.
|
||||
- `unplayable_reason` is set when `can_play` is false.
|
||||
|
||||
## `rewards`
|
||||
|
||||
```json
|
||||
{
|
||||
"state_type": "rewards",
|
||||
"rewards": {
|
||||
"items": [
|
||||
{"index": 0, "type": "gold", "description": "19 Gold", "gold_amount": 19},
|
||||
{"index": 1, "type": "potion", "description": "Energy Potion",
|
||||
"potion_id": "ENERGY_POTION", "potion_name": "Energy Potion",
|
||||
"potion_description": "Gain [ironclad_energy_icon.png][ironclad_energy_icon.png]."}
|
||||
],
|
||||
"can_proceed": true
|
||||
},
|
||||
"run": {"act": 1, "floor": 1, "ascension": 0},
|
||||
"player": { "...": "same as combat, but no hand/energy" }
|
||||
}
|
||||
```
|
||||
|
||||
- `items[]` contains **only enabled rewards**, re-indexed from 0 on every claim.
|
||||
Claim **right-to-left**, or index 0 is reclaimed forever.
|
||||
- Reward `type` values: `gold`, `potion`, `relic`, `card`, `special_card`.
|
||||
- `items` can be `[]` while `can_proceed` is true. Then the correct action is
|
||||
`proceed`.
|
||||
|
||||
## `card_reward`
|
||||
|
||||
```json
|
||||
{
|
||||
"state_type": "card_reward",
|
||||
"card_reward": {
|
||||
"cards": [{"id": "SETUP_STRIKE", "name": "Setup Strike", "type": "Attack",
|
||||
"cost": "1", "star_cost": null,
|
||||
"description": "Deal 7 damage. Gain 2 Strength this turn.",
|
||||
"rarity": "Common", "is_upgraded": false, "keywords": [], "index": 0}],
|
||||
"can_skip": true
|
||||
},
|
||||
"run": {...}, "player": {...}
|
||||
}
|
||||
```
|
||||
|
||||
**The deck is NOT exposed here.** Deck-building decisions need a composition
|
||||
snapshot taken from a combat state, which does expose all four piles.
|
||||
|
||||
## `card_select`
|
||||
|
||||
```json
|
||||
{
|
||||
"state_type": "card_select",
|
||||
"card_select": {
|
||||
"screen_type": "upgrade",
|
||||
"prompt": "Choose a card to Upgrade.",
|
||||
"cards": [{"id": "STRIKE_IRONCLAD", "name": "Strike", "type": "Attack",
|
||||
"cost": "1", "description": "Deal 6 damage.",
|
||||
"rarity": "Basic", "is_upgraded": false, "index": 0}],
|
||||
"preview_showing": true,
|
||||
"preview_cards": [
|
||||
{"name": "Strike", "description": "Deal 6 damage.", "is_upgraded": false},
|
||||
{"name": "Strike+", "description": "Deal 9 damage.", "is_upgraded": true}
|
||||
],
|
||||
"can_cancel": true,
|
||||
"can_confirm": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`preview_cards` is a before/after pair — useful for showing Jev what the
|
||||
upgrade would actually do. `screen_type` values include `upgrade`.
|
||||
|
||||
**`select_card` TOGGLES on grid screens.** Calling it twice on one index
|
||||
deselects and freezes the screen. When `preview_showing` is true and
|
||||
`can_confirm` is true, the correct action is `confirm_selection`.
|
||||
|
||||
## `event`
|
||||
|
||||
```json
|
||||
{
|
||||
"state_type": "event",
|
||||
"event": {
|
||||
"in_dialogue": false,
|
||||
"body": "…event text…",
|
||||
"options": [{"index": 0, "title": "…", "description": "…",
|
||||
"is_locked": false, "is_proceed": false, "was_chosen": false,
|
||||
"relic_name": "…", "relic_description": "…",
|
||||
"keywords": []}]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Ancient events begin with `in_dialogue: true`, which requires
|
||||
`advance_dialogue` first. Option 0 is frequently locked, which is why a blind
|
||||
`choose_event_option(index=0)` gets rejected.
|
||||
|
||||
## `rest_site`
|
||||
|
||||
```json
|
||||
{
|
||||
"state_type": "rest_site",
|
||||
"rest_site": {
|
||||
"options": [{"index": 0, "id": "…", "name": "Rest",
|
||||
"description": "…", "is_enabled": true}],
|
||||
"can_proceed": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**The field is `name`, not `title`**, plus `id` and `is_enabled`.
|
||||
|
||||
## `treasure`
|
||||
|
||||
```json
|
||||
{
|
||||
"state_type": "treasure",
|
||||
"treasure": {
|
||||
"relics": [{"index": 0, "id": "…", "name": "…", "description": "…",
|
||||
"rarity": "…", "keywords": []}],
|
||||
"can_proceed": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The chest **auto-opens**. While it opens, the response is
|
||||
`{"message": "Opening chest..."}` with **no `relics` key** and no
|
||||
`can_proceed`. Claiming then is rejected. Wait while `relics` is absent.
|
||||
|
||||
## `shop`
|
||||
|
||||
```json
|
||||
{
|
||||
"state_type": "shop",
|
||||
"shop": {
|
||||
"items": [{"index": 0, "category": "…", "price": 0,
|
||||
"is_stocked": true, "can_afford": true, "name": "…",
|
||||
"description": "…"}],
|
||||
"can_proceed": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Items carry `price`, `is_stocked`, and `can_afford` — everything needed to
|
||||
decide a purchase without extra arithmetic on the model side.
|
||||
|
||||
## `shop`
|
||||
|
||||
```json
|
||||
{
|
||||
"state_type": "shop",
|
||||
"shop": {
|
||||
"items": [
|
||||
{"index": 0, "category": "card", "price": 72, "is_stocked": true,
|
||||
"can_afford": true, "on_sale": false,
|
||||
"card_id": "STOMP", "card_name": "Stomp", "card_type": "Attack",
|
||||
"card_cost": "3", "card_rarity": "Uncommon",
|
||||
"card_description": "Deal 12 damage to ALL enemies...", "keywords": []},
|
||||
{"index": 7, "category": "relic", "price": 199, "is_stocked": true,
|
||||
"can_afford": true, "relic_id": "BLOOD_VIAL", "relic_name": "Blood Vial",
|
||||
"relic_description": "At the start of each combat, heal 2 HP."},
|
||||
{"index": 10, "category": "potion", "price": 48, "is_stocked": true,
|
||||
"can_afford": true, "potion_id": "BLOCK_POTION", "potion_name": "Block Potion",
|
||||
"potion_description": "Gain 12 Block."},
|
||||
{"index": 13, "category": "card_removal", "price": 75, "is_stocked": true,
|
||||
"can_afford": true}
|
||||
],
|
||||
"can_proceed": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**The name and description fields are category-specific:**
|
||||
|
||||
| category | name | description |
|
||||
|---|---|---|
|
||||
| `card` | `card_name` | `card_description` |
|
||||
| `relic` | `relic_name` | `relic_description` |
|
||||
| `potion` | `potion_name` | `potion_description` |
|
||||
| `card_removal` | *neither* | *neither* |
|
||||
|
||||
Reading `name`/`description` yields `None` for every category. `brain.py`
|
||||
resolves them via `shop_item_text()`.
|
||||
|
||||
A shop can offer **14+ affordable items**. Do not put them all in one `Choice`;
|
||||
see [02](02-system-one-jev.md).
|
||||
|
||||
**`can_proceed` is unreliable here** — measured `false` while `proceed()`
|
||||
worked. Never wait on it.
|
||||
|
||||
## `hand_select`
|
||||
|
||||
```json
|
||||
{
|
||||
"state_type": "hand_select",
|
||||
"hand_select": {
|
||||
"mode": "simple_select",
|
||||
"prompt": "Choose any number of cards to replace.",
|
||||
"cards": [],
|
||||
"selected_cards": [{"index": 0, "name": "Defend"}],
|
||||
"can_confirm": true
|
||||
},
|
||||
"battle": {"...": "..."}, "run": {}, "player": {}
|
||||
}
|
||||
```
|
||||
|
||||
`cards` is what is **still selectable**; `selected_cards` is what is **already
|
||||
chosen**. `combat_select_card(card_index)` indexes into the selectable cards,
|
||||
not the hand. When `cards` is empty, `combat_select_card(0)` fails with
|
||||
`Card index 0 out of range (0 selectable cards)` — the action is
|
||||
`combat_confirm_selection`.
|
||||
|
||||
## `bundle_select`
|
||||
|
||||
```json
|
||||
{
|
||||
"state_type": "bundle_select",
|
||||
"bundle_select": {
|
||||
"screen_type": "bundle",
|
||||
"prompt": "Choose a bundle.",
|
||||
"bundles": [
|
||||
{"index": 0, "card_count": 3, "cards": [
|
||||
{"id": "ANGER", "name": "Anger", "type": "Attack", "cost": "0",
|
||||
"description": "Deal 6 damage. Add a copy of this card into your Discard Pile.",
|
||||
"rarity": "Common", "is_upgraded": false, "keywords": [], "index": 0}
|
||||
]}
|
||||
],
|
||||
"preview_showing": true,
|
||||
"preview_cards": [],
|
||||
"can_cancel": true,
|
||||
"can_confirm": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Identical preview semantics to `card_select`. `select_bundle` errors with
|
||||
"A bundle preview is already open - confirm or cancel it first".
|
||||
|
||||
## Still not captured
|
||||
|
||||
Documented in `raw-simplified.md` but never observed live, so the field names
|
||||
are unverified:
|
||||
|
||||
`relic_select`, `crystal_sphere`, `fake_merchant`, `game_over`, `overlay`.
|
||||
|
||||
`brain.py` parses them defensively for that reason.
|
||||
|
||||
---
|
||||
|
||||
# Index spaces
|
||||
|
||||
**Five actions take an index. They are not the same index space.** This is the
|
||||
single most dangerous thing in the API, because the parameter names are nearly
|
||||
identical and a wrong index either hits the wrong card or silently fails.
|
||||
|
||||
| Action | Parameter | Indexes into | Notes |
|
||||
|---|---|---|---|
|
||||
| `play_card` | `card_index` | `combatState.Hand.Cards[N]` | matches `player.hand[N]` |
|
||||
| `combat_select_card` | `card_index` | `hand.ActiveHolders[N]` | **selectable cards only** |
|
||||
| `select_card` | `index` | the grid screen's card holders | matches `card_select.cards[N]` |
|
||||
| `select_card_reward` | `card_index` | the reward screen's holders | matches `card_reward.cards[N]` |
|
||||
| `shop_purchase` | `index` | the merchant inventory | matches `shop.items[N]` |
|
||||
|
||||
Verified in `McpMod.Actions.cs`:
|
||||
|
||||
```csharp
|
||||
// play_card
|
||||
var hand = player.PlayerCombatState?.Hand;
|
||||
var card = hand.Cards[cardIndex];
|
||||
|
||||
// combat_select_card
|
||||
var holders = hand.ActiveHolders;
|
||||
if (index < 0 || index >= holders.Count)
|
||||
return Error($"Card index {index} out of range ({holders.Count} selectable cards)");
|
||||
```
|
||||
|
||||
`play_card` and `combat_select_card` look like the same call. They are **not**.
|
||||
`player.hand` is every card in hand; `ActiveHolders` is only the selectable
|
||||
subset.
|
||||
|
||||
## `hand_select` has two index spaces under one name
|
||||
|
||||
```json
|
||||
{
|
||||
"cards": [{"index": 0, "name": "Defend"}],
|
||||
"selected_cards": [{"index": 0, "name": "Defend"}]
|
||||
}
|
||||
```
|
||||
|
||||
Both are called `index` and they are built from different arrays:
|
||||
|
||||
- `cards[]` is built by iterating `hand.ActiveHolders` with a fresh counter, so
|
||||
`cards[i].index == i` and indexes the **selectable** cards.
|
||||
- `selected_cards[]` is built by iterating `selectedHolders` with a **separate**
|
||||
counter, so it indexes an entirely different array.
|
||||
|
||||
Passing a `selected_cards` index to `combat_select_card` is wrong. This produced
|
||||
the observed failure:
|
||||
|
||||
```
|
||||
action rejected: Card index 0 out of range (0 selectable cards)
|
||||
```
|
||||
|
||||
## Indices shift, so the loop must be closed
|
||||
|
||||
- Playing a card removes it from hand and **renumbers every later index**.
|
||||
The mod's own notes say to play right-to-left to keep indices stable, or
|
||||
re-read between plays.
|
||||
- Claiming a reward **rebuilds and renumbers** the rewards list. Claim
|
||||
right-to-left.
|
||||
- The shop renumbers after a purchase.
|
||||
|
||||
**Rule:** observe, act ONCE, observe again. Never precompute an action list.
|
||||
|
||||
## The lesson
|
||||
|
||||
The original card-selection bug was **not** a wrong index. The index field was
|
||||
correct. The bug was choosing an index by **list position** while ignoring the
|
||||
card's own context — `name`, `type`, `rarity`, `is_upgraded` — which was present
|
||||
in the state the whole time. Since the grid lists basic Strikes first, position
|
||||
0 was always a Strike, so the bot only ever upgraded Strikes.
|
||||
|
||||
Selection must be by **identity**, and the index is only the handle used to
|
||||
address that identity.
|
||||
542
docs/research/05-failure-modes.md
Normal file
542
docs/research/05-failure-modes.md
Normal file
|
|
@ -0,0 +1,542 @@
|
|||
# 05 — Failure modes
|
||||
|
||||
Every entry below was observed **live**, caused a real infinite loop or a
|
||||
stalled run, and has a fix in the code. None are theoretical.
|
||||
|
||||
The general lesson: this interface is full of actions that report
|
||||
`{"status": "ok"}` while doing nothing. **Never trust `ok`. Trust the state.**
|
||||
|
||||
---
|
||||
|
||||
## 1. `claim_reward(index=0)` forever
|
||||
|
||||
`rewards.items[]` is rebuilt and **re-indexed from 0 after every claim**.
|
||||
Claiming index 0 repeatedly reclaims the same slot forever.
|
||||
|
||||
Observed: 65 consecutive rejected `claim_reward(index=0)` calls.
|
||||
|
||||
**Fix:** claim right-to-left, `items[-1]`.
|
||||
|
||||
## 2. A potion reward with full slots silently vanishes
|
||||
|
||||
When all potion slots are full, `claim_reward` on a potion reward returns
|
||||
`ok` and is **silently dropped**. The item never leaves `items[]`, so the loop
|
||||
never terminates.
|
||||
|
||||
Observed:
|
||||
|
||||
```
|
||||
potions: 3 of max_potion_slots 3
|
||||
claim_reward(1) -> ok | Claiming reward: potion (Energy Potion)
|
||||
items now: [gold, potion] <- unchanged
|
||||
claim_reward(0) -> ok | Claiming reward: gold (19)
|
||||
items now: [potion] <- re-indexed
|
||||
```
|
||||
|
||||
**Fix:** if the last item is a potion and `len(potions) >= max_potion_slots`,
|
||||
discard the weakest potion first, then claim. Weakness comes from a small
|
||||
`POTION_VALUE` table in `brain.py`.
|
||||
|
||||
## 3. `select_card` twice on one index freezes the screen
|
||||
|
||||
On grid screens `select_card` **toggles**. The second call deselects, so the
|
||||
state never changes and the loop stalls.
|
||||
|
||||
Observed: `select_card(index=0)` twice, then 10 unchanged reads.
|
||||
|
||||
**Fix:** when `preview_showing` and `can_confirm` are true, the action is
|
||||
`confirm_selection`, never another `select_card`.
|
||||
|
||||
## 4. A stale preview makes `confirm_selection` a no-op
|
||||
|
||||
`confirm_selection` returns `ok` while changing nothing, when the preview was
|
||||
left over from a desynchronised state.
|
||||
|
||||
Observed:
|
||||
|
||||
```
|
||||
confirm_selection -> ok "Confirming selection from preview"
|
||||
state_type before/after: card_select -> card_select
|
||||
sig changed: False
|
||||
```
|
||||
|
||||
**Fix:** if the identical `card_select` state is seen twice in a row, the
|
||||
confirm did not take effect. Send `cancel_selection` to reset, then re-select.
|
||||
Verified working:
|
||||
|
||||
```
|
||||
cancel_selection -> preview False
|
||||
select_card(0) -> preview True
|
||||
confirm_selection-> state_type becomes 'event'
|
||||
```
|
||||
|
||||
## 5. `rest_site` options use `name`, not `title`
|
||||
|
||||
Reading `options[].title` yields `None` for every option, so the handler
|
||||
concluded there were no options and fell through to a rejected `proceed`.
|
||||
|
||||
**Fix:** read `name` (plus `id`), and honour `is_enabled`.
|
||||
|
||||
## 6. `treasure` claims a relic before the chest opens
|
||||
|
||||
The chest auto-opens. During opening the response has no `relics` key and no
|
||||
`can_proceed`, so `claim_treasure_relic` is rejected repeatedly.
|
||||
|
||||
Observed: 4 consecutive rejections.
|
||||
|
||||
**Fix:** wait while `relics` is absent; claim only when present.
|
||||
|
||||
## 7. Acting during transitions
|
||||
|
||||
The loop re-decided faster than the game animated:
|
||||
|
||||
- `choose_map_node` fired **3 times in a row** during a single travel.
|
||||
- `end_turn` fired repeatedly during the enemy turn.
|
||||
|
||||
**Fix, two parts:**
|
||||
|
||||
- `__wait__` when `battle.is_play_phase` is false.
|
||||
- An **unchanged-state guard** in `run.py`: if the state signature repeats and
|
||||
the **last action succeeded**, wait. If the last action was **rejected**,
|
||||
re-decide instead, so a different action can be tried.
|
||||
|
||||
The second condition matters: waiting on a rejected action would prevent
|
||||
recovery and stall until the stuck counter fires.
|
||||
|
||||
## 8. `unknown` and `overlay` treated as dead ends
|
||||
|
||||
Both are transitions, not terminal states. Returning "stop" ended the session
|
||||
while the game was mid-transition into combat.
|
||||
|
||||
**Fix:** wait and re-observe. The unchanged-state guard bounds this, so a
|
||||
genuine dead end still stops after 10 reads.
|
||||
|
||||
## 9. The confidence gate rejected a correct answer
|
||||
|
||||
Not a loop, but a wrong decision, and worth recording because the bug was in
|
||||
**our** gate rather than in the model.
|
||||
|
||||
Measured: 5 cards offered. Jev picked `Bash` at **0.61** probability with
|
||||
`Defend` at 0.29, and reported `confidence 0.50`. A fixed 0.55 floor rejected
|
||||
it and fell back to a heuristic that chose to block instead.
|
||||
|
||||
`confidence` measures peakedness, so it falls as the option count rises:
|
||||
`(5 × 0.61 − 1) / 4 = 0.50`.
|
||||
|
||||
**Fix:** gate a `Choice` on margin over the runner-up
|
||||
(`top >= 0.45 and top - runner >= 0.20`), which is scale-free. See
|
||||
[02](02-system-one-jev.md).
|
||||
|
||||
---
|
||||
|
||||
## Checklist for a new `state_type`
|
||||
|
||||
1. Capture the real shape. Do not code from documentation.
|
||||
2. Check which fields are **absent** in some states.
|
||||
3. Ask: what does this action return when it is a no-op?
|
||||
4. Prefer `__wait__` over stopping for anything that looks transitional.
|
||||
5. Add a guard so a repeated identical state does not loop forever.
|
||||
|
||||
## Checklist for a new action
|
||||
|
||||
1. Does it report `ok` on a no-op? (Most do.)
|
||||
2. Does it re-index the collection it acts on?
|
||||
3. Is there a precondition the state exposes (`can_confirm`, `can_proceed`,
|
||||
`is_stocked`, `can_afford`) that should be checked first?
|
||||
4. Does it toggle?
|
||||
|
||||
---
|
||||
|
||||
# Session 2 additions
|
||||
|
||||
## 10. An event option ended the run, and the gate allowed it
|
||||
|
||||
Measured on a deciphering event:
|
||||
|
||||
```
|
||||
[102] jev chose Keep Deciphering conf=0.28
|
||||
[104] jev chose Lose Everything conf=0.49
|
||||
```
|
||||
|
||||
"Lose Everything" **set the player's max HP to 1**. The margin gate passed it,
|
||||
because the gate only measured how *decisive* the answer was, never how
|
||||
*consequential* the action was.
|
||||
|
||||
**Fix, in two parts:**
|
||||
|
||||
1. A stricter gate for events: `top >= 0.60 AND margin >= 0.30`.
|
||||
2. A deterministic safety net. When the model is not confident, choose the
|
||||
option with the lowest `event_safety_rank`, which counts risk words
|
||||
("everything", "keep", "continue", "gamble", "lose") minus stop words
|
||||
("stop", "leave", "refuse", "decline", "take what").
|
||||
|
||||
**A "does this risk losing the run?" Noul was tried and REMOVED.** Measured on
|
||||
the same event:
|
||||
|
||||
| Option | Risk Noul |
|
||||
|---|---|
|
||||
| "Lose Everything" | **0.46** |
|
||||
| "Keep Deciphering" | **0.52** |
|
||||
| "Stop" | 0.38 |
|
||||
|
||||
It ranked the run-ending option as *less* risky than a moderate one. A
|
||||
misleading signal is worse than no signal, so danger is detected by keywords
|
||||
instead. Keep the model for the confident case; use code for the dangerous one.
|
||||
|
||||
## 11. `hp=1/1` was reported as "healthy"
|
||||
|
||||
An effect reduced max HP to 1. `_hp_bucket` bucketed by percentage alone, so
|
||||
`1/1` was 100% and returned `healthy`. The bot walked into a normal fight at
|
||||
1 HP and died.
|
||||
|
||||
**Fix:** absolute HP is now part of the bucket. `hp <= 5` is always `critical`.
|
||||
|
||||
```python
|
||||
def _hp_bucket(pct, hp=None):
|
||||
if hp is not None and hp <= 5:
|
||||
return HP_CRITICAL
|
||||
...
|
||||
```
|
||||
|
||||
## 12. `hand_select` blindly selected index 0
|
||||
|
||||
`combat_select_card(index=0)` failed with:
|
||||
|
||||
```
|
||||
Card index 0 out of range (0 selectable cards)
|
||||
```
|
||||
|
||||
`hand_select` has `cards` (still selectable) and `selected_cards` (already
|
||||
chosen). When `cards` is empty the only useful action is
|
||||
`combat_confirm_selection`.
|
||||
|
||||
**Fix:** confirm when nothing remains selectable; otherwise give up basic
|
||||
Strikes first, then Defends.
|
||||
|
||||
## 13. `bundle_select` has the same preview trap as `card_select`
|
||||
|
||||
```
|
||||
A bundle preview is already open - confirm or cancel it first
|
||||
```
|
||||
|
||||
Same shape (`preview_showing`, `can_confirm`) and same fix: confirm when a
|
||||
preview is showing, and reset with `cancel_bundle_selection` if the identical
|
||||
state repeats.
|
||||
|
||||
## 14. Error text lives in `error`, not `message`
|
||||
|
||||
`ActionResult` read only `message`, so every rejection printed as
|
||||
`action rejected:` with nothing after it. This hid four separate bugs for a
|
||||
whole session.
|
||||
|
||||
**Fix:** `message = data.get("message") or data.get("error") or ""`.
|
||||
|
||||
**Lesson:** make failures loud before chasing them. A blank error message is
|
||||
worse than no error handling.
|
||||
|
||||
## 15. `can_proceed` is not reliable
|
||||
|
||||
For shops, `shop.can_proceed` was `false` while `proceed()` worked and moved the
|
||||
game to the map. Waiting on that flag stalls forever.
|
||||
|
||||
**Fix:** do not gate an exit on `can_proceed`. Attempt `proceed` and let the
|
||||
rejection counter bound the retries.
|
||||
|
||||
## 16. The unchanged-state guard was count-based
|
||||
|
||||
A boss death animation plus the rewards transition exceeded 10 reads, so the
|
||||
guard declared STUCK while the game was still animating.
|
||||
|
||||
**Fix:** the guard is now time-based (`--stuck-seconds`, default 25 s).
|
||||
|
||||
## 17. Transient rejections are normal
|
||||
|
||||
`proceed` at a rest site right after a heal is rejected for a moment and then
|
||||
succeeds. The retry loop was too impatient.
|
||||
|
||||
**Fix:** on rejection, back off 3x the normal pause, and allow 6 attempts.
|
||||
|
||||
---
|
||||
|
||||
# Session 3 — the programmatic audit
|
||||
|
||||
Prompted by observing that the bot "was only upgrading common attack cards".
|
||||
These are usage bugs, not decision-quality issues, and are listed separately
|
||||
from the tuning items.
|
||||
|
||||
## 18. `card_select` fallback was hardcoded to `cards[0]`
|
||||
|
||||
The upgrade screen fell back to the first card in the list, and the list is
|
||||
ordered with basic Strikes first. So every fallback upgraded a Strike.
|
||||
|
||||
Measured across runs:
|
||||
|
||||
```
|
||||
[156] [fallback] select_card(index=0) # select the first card
|
||||
[263] [fallback] select_card(index=0) # low confidence; select the first
|
||||
[378] [fallback] select_card(index=0) # low confidence; select the first
|
||||
```
|
||||
|
||||
The fallback fired often because a single `Choice` over a 13+ card deck
|
||||
dilutes, exactly like the shop.
|
||||
|
||||
**Fix, two parts:**
|
||||
|
||||
1. Re-ranking: one absolute `Noul` per card, argmax in code.
|
||||
2. Screen-aware deterministic fallback (`upgrade_rank`, `removal_rank`) that
|
||||
never picks index 0 blindly:
|
||||
- upgrade prefers a non-basic card, then Bash, then Strikes, then Defends,
|
||||
and never an already-upgraded card
|
||||
- remove/transform invert the order: shed basic Strikes and Defends first,
|
||||
and never target an upgraded card
|
||||
|
||||
Verified live after the fix:
|
||||
|
||||
```
|
||||
[197] jev chose Perfected Strike (noul=0.67)
|
||||
[205] jev chose Bludgeon (noul=0.66)
|
||||
[212] jev chose Rampage (noul=0.67)
|
||||
[303] jev chose Bash (noul=0.62)
|
||||
```
|
||||
|
||||
## 19. `hand_select` fed good cards to a "choose any number" prompt
|
||||
|
||||
The give-up ranking returned `2` for anything that was not a Strike or Defend,
|
||||
so once the basics ran out it started offering real cards:
|
||||
|
||||
```
|
||||
[023] give up Uppercut
|
||||
[086] give up Stomp
|
||||
[171] give up Bash <- the deck's only Vulnerable source
|
||||
```
|
||||
|
||||
**Fix:** only basic Strikes and Defends are candidates. When none remain,
|
||||
confirm and keep the good cards. A selection is forced only when
|
||||
`can_confirm` is false.
|
||||
|
||||
## 20. `card_reward` had a redundant gate that skipped almost everything
|
||||
|
||||
A `want_any` Noul ("does this deck want any of these?") gated the whole
|
||||
decision. When it was merely *uncertain* (0.54–0.59) the bot skipped, so it
|
||||
skipped nearly every card reward and ran a 10-card deck.
|
||||
|
||||
**Fix:** the per-card Nouls ARE the signal. Skip only when no card clears
|
||||
`CARD_PICK_THRESHOLD`.
|
||||
|
||||
## 21. A skipped card reward is NOT consumed — infinite loop
|
||||
|
||||
```
|
||||
[409] skip_card_reward() # no card cleared 0.6
|
||||
[410] claim_reward(index=2)
|
||||
[411] skip_card_reward()
|
||||
[412] claim_reward(index=2) ... forever
|
||||
```
|
||||
|
||||
Skipping returns to the rewards screen with the card **still listed**. Claiming
|
||||
it again reopens the card screen, and the cycle repeats.
|
||||
|
||||
Verified directly: `skip_card_reward` -> `ok`, then the rewards list still
|
||||
contains `[2] card: Add a card to your deck.`
|
||||
|
||||
**Fix:** record that a card reward was skipped, and ignore card rewards on the
|
||||
rewards screen afterwards. `rewards` and `card_reward` are declared one
|
||||
**screen group** so the flag survives the hop between them — otherwise it is
|
||||
cleared on every transition and the loop returns.
|
||||
|
||||
## 22. Module-level guards leaked across screens
|
||||
|
||||
Found by the new `test_brain.py`. A fresh fake-merchant shop was reported as
|
||||
"unchanged after a purchase" because a shop signature from an earlier screen
|
||||
was still set.
|
||||
|
||||
**Fix:** `_reset_screen_guards()` clears all per-screen state whenever the
|
||||
screen group changes. Also made the shop guard precise: it now only fires when
|
||||
we actually purchased from that exact shop state.
|
||||
|
||||
## 23. `fake_merchant` nests its inventory one level deeper
|
||||
|
||||
The shop is at `fake_merchant.shop.items`, not `shop.items`. Reading only
|
||||
`obs["shop"]` made every fake-merchant shop look empty, so the bot left
|
||||
immediately without buying.
|
||||
|
||||
**Fix:** resolve `obs["shop"]`, else `obs["fake_merchant"]["shop"]`, else
|
||||
`obs["fake_merchant"]`.
|
||||
|
||||
## 24. `embark` is rejected without a character selected
|
||||
|
||||
```
|
||||
action rejected: Embark button not available — select a character first
|
||||
```
|
||||
|
||||
An earlier version assumed `embark` defaults to the first unlocked character.
|
||||
It only worked once because a character happened to be selected already.
|
||||
|
||||
**Fix:** read the game's own `message`:
|
||||
- `"Select a character."` -> select one
|
||||
- `"Selected The Ironclad. Use 'confirm' to embark."` -> embark
|
||||
|
||||
## 25. `relic_select` used a diluted Choice and ignored `can_skip`
|
||||
|
||||
Verified shape: `relic_select.relics[]` with `index/id/name/description/rarity`
|
||||
plus `can_skip`. It now uses the same re-ranking pattern, honours `can_skip`,
|
||||
and falls back to the rarest relic rather than index 0.
|
||||
|
||||
## New: `test_brain.py`
|
||||
|
||||
A structural regression suite, deliberately separate from decision quality.
|
||||
It asserts:
|
||||
|
||||
1. **Every `state_type` produces an action that is LEGAL for that state.**
|
||||
Compares against `sts2.LEGAL_ACTIONS`. This is what would have caught a
|
||||
handler emitting `end_turn` during someone else's turn.
|
||||
2. **Every action the decision layer can emit is declared somewhere**, so a
|
||||
typo cannot silently produce an invalid action.
|
||||
3. **Every fallback respects its inputs** — removal does not target an
|
||||
upgraded card, upgrade does not target a basic one, card reward takes the
|
||||
rarest rather than index 0, `hand_select` gives up a Strike rather than
|
||||
Bash.
|
||||
|
||||
54 assertions, runs offline with `client=None`, no model calls, no game.
|
||||
Run it before every session.
|
||||
|
||||
---
|
||||
|
||||
# Session 4 — the toggle and transition traps
|
||||
|
||||
Found by running the bot for long stretches. Every one of these was a stall or a
|
||||
crash, not a decision-quality issue.
|
||||
|
||||
## 26. `combat_select_card` TOGGLES — re-selecting deselects
|
||||
|
||||
The same trap as `card_select`, in a different action. Measured: **18
|
||||
consecutive** `combat_select_card(card_index=0) # give up Defend` with the state
|
||||
never changing, because selecting an already-selected card deselects it.
|
||||
|
||||
`hand_select.selected_cards` is the authoritative list of what is already
|
||||
chosen. Exclude those from the candidate list.
|
||||
|
||||
**Watch out:** `cards[].index` and `selected_cards[].index` are **different
|
||||
index spaces** (different arrays, independent counters). Names are the only
|
||||
reliable way to match between them.
|
||||
|
||||
## 27. `hand_select` has a mode that only needs confirming
|
||||
|
||||
Measured state:
|
||||
|
||||
```
|
||||
mode = "upgrade_select"
|
||||
prompt = "Confirm Card to Upgrade"
|
||||
cards = [(0, "Defend")]
|
||||
selected = []
|
||||
can_confirm = true
|
||||
```
|
||||
|
||||
Sending `combat_select_card` is a silent no-op here. The card is already picked;
|
||||
`combat_confirm_selection` closes the screen and moves to `monster`.
|
||||
|
||||
**Fix:** mode-aware.
|
||||
|
||||
| mode | action |
|
||||
|---|---|
|
||||
| `upgrade_select` | `combat_confirm_selection` |
|
||||
| `simple_select` | select cards, then confirm when nothing selectable remains |
|
||||
|
||||
## 28. Character select: no indicator, and flaky embark
|
||||
|
||||
Two problems at once:
|
||||
|
||||
1. **The state carries no "selected" indicator.** The mod hardcodes
|
||||
`result["message"] = "Select a character."` whatever is chosen. Verified in
|
||||
`AddCharacterSelectMenuState`. So the message cannot be used to tell whether
|
||||
a character is picked.
|
||||
2. **Embarking right after selecting is flaky.** Measured three consecutive
|
||||
`Embark button not available - select a character first` rejections, because
|
||||
the selection had not registered yet.
|
||||
|
||||
**Fix:** ALTERNATE — select, embark, select, embark. A rejected embark is always
|
||||
followed by a fresh select, so the sequence is self-correcting regardless of
|
||||
timing.
|
||||
|
||||
An earlier attempt used signature comparison ("if the screen is unchanged, a
|
||||
character was already selected"). It failed because the signature keeps changing
|
||||
during the embark transition, restarting the cycle.
|
||||
|
||||
## 29. The run loop's guard blocked the brain's own recovery
|
||||
|
||||
`run.py` waited whenever the state was unchanged after a successful action, and
|
||||
it did so **before calling `brain.decide`**. That made it impossible for any
|
||||
handler to notice a repeated state and react differently — which is exactly how
|
||||
`character_select` and multi-line dialogue work.
|
||||
|
||||
**Fix:** move the check to AFTER deciding, and suppress only a **repeated
|
||||
action**, never acting in general. Proposing a *different* action is always
|
||||
allowed.
|
||||
|
||||
## 30. A bounded retry is required
|
||||
|
||||
Even the corrected guard stalled on Ancient dialogue: one click was issued, the
|
||||
state had not updated yet, and the guard then refused to retry forever.
|
||||
|
||||
**Fix:** `--max-duplicate-waits` (default 3). After that many suppressions on an
|
||||
unchanged state, re-execute the action. Some actions legitimately need repeating
|
||||
and some transitions are just slow.
|
||||
|
||||
## 31. `unknown` and `overlay` are transitions, not screens
|
||||
|
||||
Resetting the per-screen guards on them wiped state mid-flow. During embark the
|
||||
state flickers through `unknown`, which cleared the character-select guard and
|
||||
restarted the select/embark cycle.
|
||||
|
||||
**Fix:** `_reset_screen_guards` returns early for `unknown` and `overlay`.
|
||||
|
||||
## 32. `menu_screen` must be part of the screen key
|
||||
|
||||
`main`, `singleplayer`, `character_select` and `tutorial_prompt` all share
|
||||
`state_type == "menu"` but have completely different valid actions. Keying only
|
||||
on `state_type` meant moving from the main menu to character select did not
|
||||
reset anything.
|
||||
|
||||
**Fix:** `_screen_group()` returns `menu:<menu_screen>` for the menu state.
|
||||
|
||||
## 33. `RemoteDisconnected` crashed a run
|
||||
|
||||
```
|
||||
http.client.RemoteDisconnected: Remote end closed connection without response
|
||||
```
|
||||
|
||||
`RemoteDisconnected` is an `http.client.HTTPException`, **not** a `URLError`, so
|
||||
it escaped `jev.py`'s transport handler and killed the process mid-fight.
|
||||
|
||||
**Fix, both layers:**
|
||||
|
||||
- `jev.py` also catches `http.client.HTTPException` and `OSError`, and retries.
|
||||
- `run.py` wraps `brain.decide` in a broad `except Exception`, falls back to the
|
||||
deterministic handlers, and keeps playing.
|
||||
|
||||
## 34. Known, self-correcting: stale-state rejections
|
||||
|
||||
Occasionally the game rejects a card we chose:
|
||||
|
||||
```
|
||||
Card 'Infection' cannot be played: HasUnplayableKeyword
|
||||
Card 'Evil Eye' cannot be played: EnergyCostTooHigh
|
||||
```
|
||||
|
||||
`can_play` comes from the game's own `card.CanPlay()`, so the state was correct
|
||||
when read. A Jev call takes ~0.75 s, so the state we decided on can be about a
|
||||
second stale by the time the action lands.
|
||||
|
||||
This is a **race, not a logic bug**, and it self-corrects: the rejection sets
|
||||
`last_ok = False`, so the loop re-decides immediately against a fresh state
|
||||
instead of waiting. Cost is one wasted action. No revalidation pass is needed.
|
||||
|
||||
## Test coverage after this session
|
||||
|
||||
`test_brain.py`: **78 assertions**. New this session:
|
||||
|
||||
- indices must come from the data, never from list position (hostile-index
|
||||
cases where `index` deliberately differs from array position)
|
||||
- `hand_select` skips already-chosen cards, including duplicate names
|
||||
- `hand_select` confirms for `upgrade_select`
|
||||
- character select alternates select/embark and resets correctly
|
||||
- a `StubClient` makes the model paths testable offline and deterministically
|
||||
174
docs/research/06-decision-architecture.md
Normal file
174
docs/research/06-decision-architecture.md
Normal file
|
|
@ -0,0 +1,174 @@
|
|||
# 06 — Decision architecture
|
||||
|
||||
## The core constraint that shapes everything
|
||||
|
||||
Jev is a classifier. It picks one option from a set **we** define. It cannot
|
||||
plan, cannot invent actions, and cannot do arithmetic — and it reported 0.79
|
||||
confidence on a wrong lethal check.
|
||||
|
||||
So the architecture is not "send state, get actions". It is:
|
||||
|
||||
```
|
||||
game state ──▶ CODE computes facts and enumerates legal actions
|
||||
──▶ JEV judges between the enumerated options
|
||||
──▶ CODE gates on margin, executes exactly one action
|
||||
──▶ re-observe
|
||||
```
|
||||
|
||||
The arrow that matters is the first one. **The action space is generated by
|
||||
code, never by the model.**
|
||||
|
||||
## Three layers
|
||||
|
||||
| Layer | Engine | Responsibility | Examples |
|
||||
|---|---|---|---|
|
||||
| Facts | pure Python | arithmetic, legality, thresholds | lethal, threat bucket, block deficit, deck counts |
|
||||
| Tactics | **Jev** | preference between legal options | which card, which target, which node |
|
||||
| Gate | code | when to trust Jev | margin thresholds, fallback heuristics |
|
||||
|
||||
### Why the split is exactly here
|
||||
|
||||
Jev is good at **semantic judgement over described options**. It is bad at
|
||||
arithmetic, counting, multi-hop indirection, and large noisy states.
|
||||
|
||||
So the split puts every number in code, and every *preference* in Jev. The
|
||||
model sees conclusions:
|
||||
|
||||
```json
|
||||
{
|
||||
"combat": {
|
||||
"your_turn": true,
|
||||
"energy": 3,
|
||||
"your_health": "healthy",
|
||||
"incoming_threat": "chip",
|
||||
"lethal_available": false,
|
||||
"enemies_you_can_kill_now": "none",
|
||||
"enemies": [{"id": "NIBBIT_0", "name": "Nibbit", "hp_state": "healthy",
|
||||
"incoming": 12, "intends": "This enemy intends to Attack for 12 damage."}],
|
||||
"hand": [{"index": 2, "name": "Bash", "cost": 2, "type": "Attack",
|
||||
"targets": "AnyEnemy", "text": "Deal 8 damage. Apply 2 Vulnerable."}]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Note `your_health: "healthy"` and `incoming_threat: "chip"` — buckets, not
|
||||
numbers to compare. Note `lethal_available: false` — a conclusion computed by
|
||||
`facts.py`.
|
||||
|
||||
## Decision precedence in combat
|
||||
|
||||
```
|
||||
1. CODE: lethal proven by facts.py? -> execute the lethal line
|
||||
2. CODE: no play phase? -> wait
|
||||
3. JEV : which play is best? -> act if margin is sufficient
|
||||
4. CODE: fallback heuristic -> act
|
||||
```
|
||||
|
||||
Step 1 never consults the model. That is the direct consequence of the
|
||||
arithmetic failure: `facts.py` proves lethal and a deterministic search
|
||||
executes it. Across 759 loop steps, **33 lethal lines were executed by code**
|
||||
and Jev was never asked "can I kill this".
|
||||
|
||||
Step 4 exists because a low-confidence answer must not become a guess. The
|
||||
fallback is a documented heuristic adapted from the STS2MCP strategy notes.
|
||||
|
||||
## Current coverage
|
||||
|
||||
| Decision | Jev | Fallback | Status |
|
||||
|---|---|---|---|
|
||||
| Combat: card play + target | yes | heuristic | working |
|
||||
| **Combat: potion use** | yes, plus a hard-need override | spend when lethal | working |
|
||||
| Map pathing | yes | HP / gold heuristic | working |
|
||||
| Card reward | yes (with `skip`) | take first | falls back often |
|
||||
| Card select: upgrade/transform/remove | yes | select first | working |
|
||||
| Event option | yes, stricter gate | keyword safety net | working |
|
||||
| Relic select | yes | take first | shape unverified |
|
||||
| **Shop purchase** | yes (re-ranking) | leave | working |
|
||||
| **Treasure relic** | yes when >1 offered | take first | working |
|
||||
| **Bundle select** | yes | first bundle | working |
|
||||
| In-combat exhaust/discard select | **no** | Strikes, then Defends | heuristic |
|
||||
| Rest site: heal vs upgrade | **no** | HP < 60% → heal | heuristic |
|
||||
| Crystal sphere | **no** | skip | gap |
|
||||
|
||||
Combat now issues three kinds of action: `play_card`, `use_potion`, and
|
||||
`end_turn`.
|
||||
|
||||
### Potion precedence, and why it is split
|
||||
|
||||
```
|
||||
1. CODE: lethal by cards? -> play the lethal line
|
||||
2. CODE: not play phase? -> wait
|
||||
3. CODE: incoming hit lethal AND no card can prevent it
|
||||
-> spend a potion (no model)
|
||||
4. JEV : is a potion worth spending now? -> use it if confident
|
||||
5. JEV : which play is best? -> play it
|
||||
6. CODE: fallback heuristic -> act
|
||||
```
|
||||
|
||||
Step 3 exists because Jev answered the soft potion question at **0.61**, just
|
||||
under the 0.65 Noul floor, on a state where the next hit was lethal and no card
|
||||
could prevent it. Delegating that decision would have lost the run. Code
|
||||
decides **that** a potion must be spent; Jev decides **which**.
|
||||
|
||||
The gaps cluster in **resource spending** — potions, gold, and one-shot card
|
||||
effects. That is the category that decides boss fights. Run 1 died to Vantom
|
||||
holding all three potions, which is a direct consequence of the potion gap.
|
||||
|
||||
## Batching
|
||||
|
||||
All questions for one state go in **one** call. Questions are evaluated in
|
||||
parallel, and measured latency barely moves with question count:
|
||||
|
||||
| Request | Time |
|
||||
|---|---|
|
||||
| 1 short question | 0.73 s |
|
||||
| 3 questions, full combat state | 0.90 s |
|
||||
|
||||
So adding a potion question to the combat call costs **no extra latency**.
|
||||
A question that is only sometimes relevant is still worth asking
|
||||
(speculative fan-out); the code ignores answers it does not need.
|
||||
|
||||
Questions in one call are independent. If Q2 needs Q1's answer, use a second
|
||||
call.
|
||||
|
||||
## The closed loop
|
||||
|
||||
```
|
||||
observe -> decide -> execute ONE action -> observe again
|
||||
```
|
||||
|
||||
Strictly closed, because **playing a card removes it from hand and shifts every
|
||||
later index**. An action list computed up front would be wrong after the first
|
||||
play. The same applies to reward lists, which re-index on every claim.
|
||||
|
||||
## Confidence gating
|
||||
|
||||
| Answer type | Gate |
|
||||
|---|---|
|
||||
| `Choice` | `top >= 0.45 and (top - runner_up) >= 0.20` |
|
||||
| `Noul` | `abs(noul - 0.5) >= 0.15` |
|
||||
| `Score` | `confidence >= threshold` |
|
||||
|
||||
Never gate a `Choice` on `confidence` alone. It measures peakedness and
|
||||
therefore falls as the option count rises.
|
||||
|
||||
## Robustness rules
|
||||
|
||||
1. Never trust `status: ok`. Verify against the next state read.
|
||||
2. Wait on transitions; do not stop.
|
||||
3. Bound every wait, and bound consecutive rejections.
|
||||
4. When the state repeats after a **successful** action, wait.
|
||||
When it repeats after a **rejected** action, try something else.
|
||||
5. Claim and select collections **right-to-left** when they re-index.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Can a System One policy clear an act boss at all, or does the macro layer
|
||||
need escalation to a reasoning model? Run 1 reached floor 16, so the plateau
|
||||
is now measurable rather than assumed.
|
||||
- Card reward fell back to "take the first" several times on thin margins.
|
||||
Composite `Score` questions per axis, combined with weights in code, may
|
||||
beat a single `Choice`.
|
||||
- Deck tracking is currently a composition snapshot persisted to `deck.json`,
|
||||
refreshed from combat states. It will drift if cards are removed outside
|
||||
combat without an intervening fight.
|
||||
206
docs/research/07-run-log.md
Normal file
206
docs/research/07-run-log.md
Normal file
|
|
@ -0,0 +1,206 @@
|
|||
# 07 — Run log
|
||||
|
||||
Results per run. Source of truth is the game's own history file:
|
||||
|
||||
```
|
||||
~/Library/Application Support/SlayTheSpire2/steam/<steamid>/modded/profile1/saves/history/<ts>.run
|
||||
```
|
||||
|
||||
That file records `win`, `killed_by_encounter`, `seed`, `run_time`, `build_id`,
|
||||
`ascension`, `was_abandoned`, and the full deck. **It is the evaluation
|
||||
harness.** Read it rather than instrumenting the bot.
|
||||
|
||||
---
|
||||
|
||||
## Run 001 — Ironclad, standard, ascension 0
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Outcome | **Loss** |
|
||||
| Killed by | `ENCOUNTER.VANTOM_BOSS` (Act 1 boss) |
|
||||
| Floor reached | 16 |
|
||||
| Seed | `JUS59Z32HF` |
|
||||
| Run time | 2017 s (33 min) |
|
||||
| Game build | `v0.107.1` |
|
||||
| Abandoned | no |
|
||||
|
||||
### Progress
|
||||
|
||||
- Cleared 15 floors.
|
||||
- Beat an elite: **Byrdonis**, 81 HP, 17-damage attack, `Territorial` (+1
|
||||
Strength each turn).
|
||||
- Fought Nibbits, Twig Slimes, Leaf Slimes, Shrinker Beetle, Inklings, Mawler,
|
||||
Ruby Raiders (3-enemy fight), Vantom.
|
||||
- Healed at rest sites at 43% and 24% HP.
|
||||
|
||||
### Final deck (19 cards, 1 upgraded)
|
||||
|
||||
`5× Strike` (one at upgrade level 1), `4× Defend`, `Bash`, `Setup Strike`,
|
||||
`Inflame`, `Iron Wave`, `Rage`, `Stomp`, `Bludgeon`, `Battle Trance`,
|
||||
`Second Wind`, `Bloodletting`.
|
||||
|
||||
### Relics (3)
|
||||
|
||||
`Burning Blood` (floor 1), `Gorget` (floor 10), `Vajra` (floor 11).
|
||||
|
||||
### Potions held at death — all three
|
||||
|
||||
```
|
||||
POTION.EXPLOSIVE_AMPOULE slot 0
|
||||
POTION.STRENGTH_POTION slot 1
|
||||
POTION.ENERGY_POTION slot 2
|
||||
```
|
||||
|
||||
**This is the headline finding.** The combat brain had no potion logic, so the
|
||||
bot died holding an Explosive Ampoule, a Strength Potion, and an Energy Potion.
|
||||
The STS2MCP strategy notes state the principle directly: *"Don't hoard potions.
|
||||
Dying with full potions is the worst outcome."*
|
||||
|
||||
The run was lost on tactics, not on deck quality. The deck was reasonable for
|
||||
floor 16 and the relic set was functional.
|
||||
|
||||
### Decision sources
|
||||
|
||||
| Source | Count |
|
||||
|---|---|
|
||||
| `code` | 141 |
|
||||
| `jev` | 70 |
|
||||
| `fallback` | 35 |
|
||||
| Lethal lines executed by code | 33 |
|
||||
|
||||
Aggregate across 759 loop steps in 5 sessions.
|
||||
|
||||
### Notable Jev calls
|
||||
|
||||
| Situation | Jev chose | Confidence |
|
||||
|---|---|---|
|
||||
| Combat, 5 cards | Bash over Defend | 0.85 |
|
||||
| Card reward | Inflame | 0.28 |
|
||||
| Card reward | Rage | 0.40 |
|
||||
| Card reward | Setup Strike | 0.33 |
|
||||
| Map, 2 options | Shop | 0.97 |
|
||||
| Map, 2 options | RestSite | 0.88 |
|
||||
| Boss fight | Battle Trance, Bloodletting, Second Wind | 0.62–1.00 |
|
||||
|
||||
Several card-reward margins were thin enough that the gate fell back to "take
|
||||
the first". Card reward is the weakest Jev decision in the current design.
|
||||
|
||||
### Bugs this run exposed
|
||||
|
||||
Eight infinite loops or stalls, all now fixed. See
|
||||
[05-failure-modes.md](05-failure-modes.md). The most expensive were the
|
||||
right-to-left reward indexing and the silently-dropped potion reward.
|
||||
|
||||
---
|
||||
|
||||
## Baseline to beat
|
||||
|
||||
| Metric | Run 001 |
|
||||
|---|---|
|
||||
| Win | no |
|
||||
| Floor | 16 |
|
||||
| Act 1 boss reached | yes |
|
||||
| Act 1 boss killed | no |
|
||||
|
||||
Run 002 should be measured against this. The first question is not "does it
|
||||
win" but "does it get past Vantom, and does it die holding potions again".
|
||||
|
||||
## Characters unlocked
|
||||
|
||||
| Character | Status |
|
||||
|---|---|
|
||||
| The Ironclad | unlocked |
|
||||
| The Silent | unlocked (after the `NEOW_EPOCH` reveal) |
|
||||
| The Regent | locked |
|
||||
| The Necrobinder | locked |
|
||||
| The Defect | locked |
|
||||
|
||||
`progress.save` recorded `pending_character_unlock: "CHARACTER.SILENT"` before
|
||||
the reveal, then Silent became selectable.
|
||||
|
||||
## Seeded runs
|
||||
|
||||
Standard singleplayer exposes **no seed**, and `run` state carries no seed
|
||||
field. The seed is visible only in the history file, after the run.
|
||||
|
||||
Seeded modes exist but were not available until the first epoch unlock:
|
||||
|
||||
| Mode | Seeded |
|
||||
|---|---|
|
||||
| `standard` | no |
|
||||
| `daily` | yes (fixed per day) |
|
||||
| `custom` | yes |
|
||||
|
||||
For a reproducible evaluation harness, `custom` is the likely path. This needs
|
||||
verification — the mode screen appeared once and was not captured.
|
||||
|
||||
---
|
||||
|
||||
# Session 2 — potions, shops, treasure, events
|
||||
|
||||
## Scoreboard
|
||||
|
||||
| # | Win | Killed by | Seed | Time | Potions at end | Act reached |
|
||||
|---|---|---|---|---|---|---|
|
||||
| 001 | no | `VANTOM_BOSS` | `JUS59Z32HF` | 2017 s | **3** | 1 boss |
|
||||
| 002 | no | `BYRDONIS_ELITE` | `MJ4J25A3BH` | 688 s | 0 | 1 |
|
||||
| 003 | no | `SNAPPING_JAXFRUIT_NORMAL` | `HKLFARJJJQ` | 186 s | 0 | 1 |
|
||||
| 004 | no | **`THE_INSATIABLE_BOSS`** | `MLQ4KBZQWV` | 2235 s | 2 | **2 boss** |
|
||||
|
||||
Run 004 reached the Act 2 boss with 11 relics and a 29-card deck. That is the
|
||||
best result so far and the first time the bot cleared an act.
|
||||
|
||||
## Run 003 — the catastrophic event
|
||||
|
||||
Died to a **normal** encounter at 186 s. The cause was not combat:
|
||||
|
||||
```
|
||||
[102] jev chose Keep Deciphering conf=0.28
|
||||
[104] jev chose Lose Everything conf=0.49
|
||||
```
|
||||
|
||||
The event offered "Keep Deciphering" and "Lose Everything". Jev picked
|
||||
"Lose Everything", which **set the player's max HP to 1**. Every subsequent
|
||||
state read then showed:
|
||||
|
||||
```
|
||||
hp=1/1 (healthy)
|
||||
```
|
||||
|
||||
The bot believed it was at full health, walked into a normal fight at 1 HP, and
|
||||
died. Two separate defects, both now fixed:
|
||||
|
||||
1. `_hp_bucket` bucketed by percentage only, so `1/1` was 100% = "healthy".
|
||||
Fixed: absolute HP <= 5 is always `critical`.
|
||||
2. The event gate accepted a run-ending option at 0.49. Fixed with a stricter
|
||||
event gate plus a deterministic keyword safety net. See
|
||||
[05](05-failure-modes.md) and [02](02-system-one-jev.md).
|
||||
|
||||
## Run 004 — what the new decision loops did
|
||||
|
||||
| Decision | Count | Notes |
|
||||
|---|---|---|
|
||||
| `shop_purchase` | 6 in one shop | Lantern, Feel No Pain, Evil Eye, Salvo, Equilibrium, Headbutt |
|
||||
| `use_potion` | 2+ | Fire Potion chosen by Jev at conf 1.00; two spent on lethal hits |
|
||||
| `claim_treasure_relic` | 2 | Lucky Fysh, plus earlier Orichalcum and Bronze Scales |
|
||||
| `combat_confirm_selection` | 4 | Hand-select now confirms instead of failing |
|
||||
| event safety net | 1 | "uncertain (0.44); took safest option" |
|
||||
|
||||
Gold went from 528 to 130 across Act 2, so the "never buy anything" gap is
|
||||
closed. Potions at death dropped from 3 (run 001) to 2 (run 004) while the run
|
||||
went a full act further.
|
||||
|
||||
## Still open
|
||||
|
||||
- The bot still died holding 2 potions. The hard-need path only fires when the
|
||||
incoming hit is *lethal*; it does not yet spend potions on a losing
|
||||
attrition fight where HP is dropping every turn.
|
||||
- Buying 6 items in one shop produced a 29-card deck. `SHOP_BUY_THRESHOLD`
|
||||
may be too permissive; deck dilution is a real cost.
|
||||
- Card reward still falls back to "take the first" fairly often.
|
||||
|
||||
## Recurring manual blocker
|
||||
|
||||
The Timeline epoch reveal blocked the menu again after run 004, this time with
|
||||
`IRONCLAD2_EPOCH`. The mod refuses to automate it. A human must reveal it
|
||||
before the next run can start. Expect this after most runs.
|
||||
74
docs/research/README.md
Normal file
74
docs/research/README.md
Normal file
|
|
@ -0,0 +1,74 @@
|
|||
# STS2 Bot — Research Notes
|
||||
|
||||
Living notes on reverse-engineering Slay the Spire 2 and driving it with a
|
||||
TypeSafe System One (Jev) decision model.
|
||||
|
||||
These documents record **what was measured**, not what was assumed. Where a
|
||||
claim comes from a live test, the evidence is quoted. Where something is a
|
||||
guess, it says so.
|
||||
|
||||
## Contents
|
||||
|
||||
| # | Document | Covers |
|
||||
|---|---|---|
|
||||
| 01 | [Game engine and mod surface](01-game-engine-and-mod-surface.md) | Engine, assemblies, the official mod loader, manifest schema |
|
||||
| 02 | [System One / Jev](02-system-one-jev.md) | What Jev is, measured latency and cost, the arithmetic failure |
|
||||
| 03 | [STS2MCP interface](03-sts2mcp-interface.md) | The community mod, version drift, rebuilding from source |
|
||||
| 04 | [State shapes](04-state-shapes.md) | Every verified JSON shape, per `state_type` |
|
||||
| 05 | [Failure modes](05-failure-modes.md) | Every infinite loop found live, with its fix |
|
||||
| 06 | [Decision architecture](06-decision-architecture.md) | The three-layer design and why the split is where it is |
|
||||
| 07 | [Run log](07-run-log.md) | Results per run, with seeds and outcomes |
|
||||
|
||||
Architecture summary lives in [`../DESIGN.md`](../DESIGN.md).
|
||||
|
||||
## How to add a finding
|
||||
|
||||
1. Prefer a measurement over an inference. Quote the command and the output.
|
||||
2. Put game facts in 01/03/04, model facts in 02, bugs in 05.
|
||||
3. Record the **date** and the **game build** (`v0.107.1` today). Both move.
|
||||
4. When a finding is later disproved, do not delete it. Mark it superseded
|
||||
and say what replaced it. The wrong turn is often the useful part.
|
||||
|
||||
## Environment these notes were taken on
|
||||
|
||||
| Item | Value |
|
||||
|---|---|
|
||||
| Game build | `v0.107.1`, commit `59260271` |
|
||||
| Platform | macOS (arm64), Steam |
|
||||
| Engine | Godot 4.5.1 (.NET), runtime .NET 9.0.7 |
|
||||
| Mod | STS2MCP, rebuilt from upstream `main` @ `55e0648` |
|
||||
| Model | `jev-latest` resolving to `jev-1.13.0` |
|
||||
|
||||
## Rebuilding the mod
|
||||
|
||||
`STS2MCP` release `0.4.0` is **broken on this game build**. See
|
||||
[03](03-sts2mcp-interface.md). To rebuild:
|
||||
|
||||
```bash
|
||||
cd ~/sts2-bot/vendor/STS2MCP
|
||||
nix shell nixpkgs#dotnet-sdk_9 --command bash -c '
|
||||
dotnet build STS2_MCP.csproj -c Release -o out/STS2_MCP \
|
||||
-p:STS2GameDir="$HOME/Library/Application Support/Steam/steamapps/common/Slay the Spire 2"'
|
||||
cp out/STS2_MCP/STS2_MCP.dll \
|
||||
"$HOME/Library/Application Support/Steam/steamapps/common/Slay the Spire 2/SlayTheSpire2.app/Contents/MacOS/mods/"
|
||||
```
|
||||
|
||||
Then restart the game. Mods load only at process start.
|
||||
|
||||
## Testing
|
||||
|
||||
Two suites, both offline. Run them before every session.
|
||||
|
||||
```bash
|
||||
python3 test_brain.py # 54 assertions — structural / programmatic
|
||||
python3 test_facts.py # 29 assertions — arithmetic and parsing
|
||||
```
|
||||
|
||||
`test_brain.py` is the important one for catching usage bugs. It asserts that
|
||||
every `state_type` produces an action **legal for that state**, that every
|
||||
action the decision layer can emit is declared somewhere, and that every
|
||||
fallback respects its own inputs. It needs no model and no running game.
|
||||
|
||||
It was added after a session in which four separate infinite loops and three
|
||||
hardcoded fallbacks were found by hand. Most of them would have been caught
|
||||
here.
|
||||
Loading…
Add table
Add a link
Reference in a new issue