sts2-bot/docs/DESIGN.md
0xrsydn fb32822468 Add design doc and research notes
DESIGN.md covers the three-layer architecture (facts in code, Jev for
tactics, gated escalation for macro). research/ documents the engine and
mod surface, the Jev classifier's measured behavior, the STS2MCP HTTP
interface, state shapes, failure modes, decision architecture, and a
run log of the first four sessions.
2026-09-22 00:01:22 +07:00

16 KiB

STS2 Bot — Design (verified 2026)

Jev (TypeSafe System One) as the tactics policy. Confidence-gated LLM escalation for macro.

Verified environment

Item Value
Game build v0.107.1, commit 59260271
Engine Godot 4.5.1 (.NET), .NET 9.0.7, osx-arm64
Main assembly sts2.dll + sts2.xml (19,635 documented members)
Mods dir SlayTheSpire2.app/Contents/MacOS/mods/ (absent)
Workshop dir Steam/steamapps/workshop/content/2868840/ (absent)
TypeSafe API live; needs TYPESAFE_API_KEY (unset)
STS2MCP v0.4.0, tested against game v0.103.2
Python SDKs none installed; Node available

Manifest schema (JSON, from STS2_MCP.json)

{"id","name","author","description","version","has_pck","has_dll","affects_gameplay"}

Note: snake_case in JSON, not the C# property names.

Actuator: STS2MCP

Base URL http://localhost:15526. No auth.

  • GET /api/v1/singleplayer?format=json|markdown — read state
  • POST /api/v1/singleplayer{"action": ..., ...}
  • GET /api/v1/compendium, /profile, /wiki?query=, /profiles

Response always has state_type, run{act,floor,ascension}, player.

menu, game_over      -> menu_select
monster/elite/boss   -> play_card, use_potion, end_turn
hand_select          -> combat_select_card, combat_confirm_selection
rewards              -> claim_reward, proceed
card_reward          -> select_card_reward, skip_card_reward
map                  -> choose_map_node
event                -> choose_event_option, advance_dialogue
rest_site            -> choose_rest_option, proceed
shop/fake_merchant   -> shop_purchase, proceed
treasure             -> claim_treasure_relic, proceed
card_select          -> select_card, confirm_selection, cancel_selection
bundle_select        -> select_bundle, confirm_bundle_selection, cancel_bundle_selection
relic_select         -> select_relic, skip_relic_selection
crystal_sphere       -> crystal_sphere_set_tool/click_cell/proceed
overlay, unknown     -> none (manual)

use_potion / discard_potion work in any state where potions are reachable.

Card rules text ships in game state when cards are visible.

Decision loop

Card indices shift on every play. The loop is strictly closed:

observe -> compute facts (pure code) -> ask Jev (1 batched call)
        -> gate on confidence -> execute ONE action -> re-observe

Never precompute an action list.

Three layers

Layer Engine Responsibility
Facts pure Python arithmetic, legality, deck stats, threat model
Tactics Jev per-turn card play, targeting, potions, event options
Macro scripted -> LLM archetype, card rewards, routing, boss prep
Gate confidence threshold when to escalate to Claude/GPT

Critical: Jev cannot count or do arithmetic (documented jaggedness). Compute numbers in code. Send conclusions, not raw values.

Jev primitives

  • Choice: pick 1 of <=255 options -> choice, probabilities, confidence
  • Score: position on ordered levels -> score, legend, probabilities, confidence
  • Noul: yes/no -> noul (probability only, no confidence)

Batch all questions for one state into ONE call. Parallel, ~100ms, near-free latency, output tokens free. 12.2x cheaper than separate calls.

Limits: 64k context total; 32k state + longest question. Text only. Price $0.042/Mtok input. Rate 250k tok/s, 1200 req/min.

Combat question template (one batched call)

lethal_this_turn  Noul    any line in hand kills all enemies
must_block        Noul    preventing damage beats dealing damage
best_play         Choice  one option per legal play, described in words
target            Choice  which enemy to hit

Precedence in code: lethal -> must_block -> best_play. Confidence < threshold -> fall back to AGENTS.md heuristics.

Cost projection

Metric Estimate
Tokens/run ~2M filtered
Cost/run ~$0.08-0.35
Model time/run ~100-150 s
Wall clock/run ~5-15 min
Frontier LLM agent (reported) ~8M tok, ~$20-40/run

Risks

  1. Version drift: STS2MCP tested on v0.103.2, game is v0.107.1.
  2. Jev arithmetic/counting failure -> mitigate by computing in code.
  3. Context rot -> send filtered state, never a raw dump.
  4. Index shift -> strictly closed loop, re-read after each action.
  5. Macro plateau -> confidence-gated escalation to a reasoning LLM.
  6. Doc drift: repo AGENTS.md uses MCP tool names (combat_end_turn, rewards_pick_card); HTTP uses (end_turn, select_card_reward). Trust raw-simplified.md for direct HTTP use.

MEASURED RESULTS (validated against live API)

Credential

Key found at ~/.config/secrets/global-env/TYPESAFEAI_API_KEY (sops-nix symlink -> ~/.config/sops-nix/secrets/). NOT in the shell environment. Read from that path at runtime; never log the value.

Validation

First call: HTTP 200, model: jev-1.13.0. Game-correct answers on a combat scenario (Cleave into the 12 HP attacker, confidence 0.85).

Latency (measured, NOT the documented 100 ms)

Request Time
Minimal (1 short noul) 0.73 s
Minimal, 5 repeats 0.73 / 0.73 / 0.74 / 0.80 / 0.76 s
Full combat state, 3 questions 0.90 s
GET /v1/models ttfb 0.68 s (connect 0.22 s)

Floor is ~0.73 s, dominated by network + server TTFB. State size and question count are nearly free -> fan-out is confirmed.

Revised budget: ~0.75 s per action call. ~500-600 actions per run -> ~7-8 min pure API time. Wall clock per run: ~20-45 min.

Arithmetic failure CONFIRMED (critical)

Ground truth: energy 3, hand Strike(1c/6dmg) x3 + Bash(2c/8dmg), target 19 HP. Max reachable damage = 18. Lethal = NO.

max_damage_bucket -> "18_to_23"  (0.73)   CORRECT
lethal_available  -> 0.79                 WRONG (true answer: no)

The model bucketed the magnitude correctly but failed the threshold comparison (18 vs 19).

CRITICAL IMPLICATION: confidence was 0.79 on a wrong answer. Confidence gating does NOT protect against arithmetic errors. All arithmetic (lethal, block deficit, damage totals) MUST be computed in code and passed in as conclusions. Never ask Jev to compare numbers.

Bundle signing (macOS)

App is adhoc-signed; spctl -a already returns "rejected". No quarantine xattr (only com.apple.provenance). Adding mod files breaks the seal but there is no enforced signature. Rollback: rm -rf <bundle>/Contents/MacOS/mods.

Installed

STS2MCP v0.4.0 -> SlayTheSpire2.app/Contents/MacOS/mods/ (staged copy kept in ~/sts2-bot/vendor/) Mods load at startup, so the game must be restarted.

VERSION DRIFT: STS2MCP v0.4.0 is broken on game v0.107.1

Symptom

GET /api/v1/singleplayer returns HTTP 500 once combat starts:

"error": "Failed to read game state: Method not found:
          'Boolean MegaCrit.Sts2.Core.Combat.CombatManager.get_IsPlayPhase()'."
"exception_type": "System.MissingMethodException"
at STS2_MCP.McpMod.BuildBattleState(RunState, CombatRoom)

Menu, map, and character-select states work. COMBAT state fails.

Root cause

CombatManager.IsPlayPhase was removed in game v0.107.1. Verified: grep -c IsPlayPhase sts2.xml -> 0.

Replacement in v0.107.1 (confirmed in sts2.xml):

  • MegaCrit.Sts2.Core.Combat.PlayerTurnPhase.Play (per-player)
  • PlayerCombatState.Phase
  • ActionSynchronizerCombatState.PlayPhase / .NotPlayPhase
  • CombatManager.DebugOnlyGetState().CurrentSide == CombatSide.Player

Timeline

  • release 0.4.0 2026-05-05 (broken, pre-fix)
  • fix commit 55e06485 2026-07-29 PR #123 "Fix game API compatibility with STS2 v0.107"
  • no tag after 0.4.0, no CI workflow, no fork ships a prebuilt DLL

Resolution: build from source

Installed .NET 9 SDK ephemerally via nix (no system changes):

nix eval --raw nixpkgs#dotnet-sdk_9.name   -> dotnet-sdk-wrapped-9.0.310

Build (macOS):

cd ~/sts2-bot/vendor/STS2MCP
nix shell nixpkgs#dotnet-sdk_9 --command bash -c '
  dotnet build STS2_MCP.csproj -c Release -o out/STS2_MCP \
    -p:STS2GameDir="$HOME/Library/Application Support/Steam/steamapps/common/Slay the Spire 2"'

Result: Build succeeded, 0 warnings, 0 errors. Installed 236,544 B dll (release was 194,560 B). Old release dll kept at vendor/STS2_MCP.dll.release-0.4.0.

Notes

  • Upstream open issues #131/#132 track v0.111 compatibility. Re-check before upgrading the game, since the same class of break will recur.
  • The csproj resolves the data dir on macOS automatically to SlayTheSpire2.app/Contents/Resources/data_sts2_macos_arm64 and references sts2.dll, GodotSharp.dll, 0Harmony.dll with Private=false.

Confirmed save layout (modded)

~/Library/Application Support/SlayTheSpire2/steam/<steamid>/
    settings.save            {"mod_settings": {"mods_enabled": true}}
    profile.save
    profile1/saves/...       vanilla progress
    modded/profile1/saves/   MODDED progress (isolated)
        prefs.save, progress.save, history/

Modded runs never touch vanilla saves.

Confirmed state shapes

menu (main): {state_type, menu_screen, message, options[]} character_select: {state_type, menu_screen, message, characters[]{name,id,locked,hp,gold,energy, description,starting_relics[],starting_deck[],total_cards,total_relics,total_potions}, options[]{name,enabled}} Only IRONCLAD unlocked on a fresh profile. No seed option in standard SP (docs: supplying seed here errors). tutorial_prompt: {state_type, menu_screen, message, options[]{name,enabled}} -> choose 'no' map: {state_type, map{visited,next_options[]{index,col,row,type,leads_to},nodes[]}, run{act,floor,ascension}, player{character,hp,max_hp,block,gold,status[],relics[],potions[],max_potion_slots}} No seed field is exposed in run state. combat: BLOCKED until the rebuilt dll loads.

MILESTONE: bot reached the Act 1 boss

First full run result (run history file)

~/Library/Application Support/SlayTheSpire2/steam/<id>/modded/profile1/saves/history/<ts>.run

win                    : false
killed_by_encounter    : "ENCOUNTER.VANTOM_BOSS"
seed                   : "JUS59Z32HF"
run_time               : 2017 s
build_id               : "v0.107.1"
ascension              : 0
game_mode              : "standard"
was_abandoned          : false
acts                   : 3
schema_version         : 9

The bot cleared 15 floors, beat an elite (Byrdonis, 81 HP), healed at rest sites, and died to the Act 1 boss Vantom on floor 16.

Evaluation harness: SOLVED

The .run history file IS the eval record. It exposes win/loss, killed_by, seed, run_time, and build_id. Stage 5 (win rate over seeds) reads this file. The seed is only visible HERE -- it is not in live state and cannot be set for standard singleplayer.

Decision sources (759 loop steps, 5 sessions)

code     141   deterministic: lethal lines, procedure, arithmetic
jev       70   preference: which card, which node, which event option
fallback  35   heuristic: low-confidence recovery, no-Jev mode
lethal lines executed by code: 33

State shapes discovered (all verified live)

menu           {state_type, menu_screen, message, options[],
                blocked_options[]{name,enabled,reason,pending_epoch_ids}}
character_select {characters[]{name,id,locked,hp,gold,energy,description,
                starting_relics[],starting_deck[],total_cards,...}, options[]}
map            {map{visited,next_options[]{index,col,row,type,leads_to},nodes[],
                boss{id,name}}, run, player}
monster/elite/boss
               {battle{round,turn,is_play_phase,enemies[]{entity_id,combat_id,
                name,hp,max_hp,block,status[]{id,name,amount,type,description},
                intents[]{type,label,title,description}}},
                run{act,floor,ascension},
                player{character,hp,max_hp,block,energy,max_energy,hand[],
                draw_pile[],draw_pile_count,discard_pile[],exhaust_pile[],
                gold,status[],relics[],potions[],max_potion_slots}}
  card  : {id,name,type,cost(str),star_cost,description,rarity,is_upgraded,
           keywords[],index,target_type,can_play,unplayable_reason}
  pile card: {name,cost,star_cost,description}   <-- NO id/type/index
  intent label: damage as a STRING, e.g. "12"; may be "12x2"
rewards        {rewards{items[]{index,type,description,[gold_amount],
                [potion_id,potion_name,potion_description]}, can_proceed}}
                items[] contains ONLY enabled rewards, re-indexed per claim
card_reward    {card_reward{cards[], can_skip}}   <-- NO deck composition
card_select    {card_select{screen_type,prompt,cards[],preview_showing,
                preview_cards,can_cancel,can_confirm}}
event          {event{in_dialogue,body,options[]{index,title,description,
                is_locked,is_proceed,was_chosen,[relic_name,relic_description]}}}
rest_site      {rest_site{options[]{index,id,name,description,is_enabled},
                can_proceed}}          <-- `name`, NOT `title`
treasure       {treasure{relics[]{index,id,name,description,rarity},
                can_proceed,message}}  <-- relics ABSENT while chest opens
shop           {shop{items[],can_proceed}}

Bugs found live and fixed (all were real infinite loops)

  1. claim_reward(index=0) forever. items[] re-indexes on every claim. Must claim RIGHT-TO-LEFT.
  2. claim_reward on a potion with all slots full. Returns "ok" and is SILENTLY DROPPED, so the item never leaves the list. Fix: discard the weakest potion first (POTION_VALUE table in brain.py).
  3. select_card twice on one index. It TOGGLES on grid screens; the second call deselects and freezes the screen. Fix: if preview_showing and can_confirm -> confirm_selection.
  4. Stale preview: confirm_selection returns ok and changes nothing. Fix: if the identical card_select state repeats, cancel_selection to reset.
  5. rest_site read options[].title, but the field is name (+ is_enabled). Fix applied.
  6. treasure claimed a relic before the chest auto-opened. relics[] is absent during opening; only can_proceed is set. Fix: wait while relics absent.
  7. Acting during transitions. choose_map_node fired 3x during travel; end_turn during the enemy turn. Fix: wait on not-in-play-phase, plus an unchanged-state guard in run.py that waits when the last action LANDED, and re-decides when it was REJECTED.
  8. state_type 'unknown'/'overlay' treated as a dead end. These are transitions. Fix: wait instead of stopping.

Confidence gate bug (found by measurement)

A fixed confidence floor is wrong when option count varies. Measured: 5 cards, Jev picks Bash at 0.61 with Defend at 0.29 -> confidence 0.50 because confidence measures peakedness: (5*0.61-1)/4 = 0.50. A 0.55 floor rejected a clear plurality.

Fix: gate a Choice on (top >= 0.45 AND top - runner_up >= 0.20). Noul has no confidence; gate on |noul-0.5| >= 0.15.

BLOCKER: Timeline epoch reveal needs a human

After the run ended, the main menu offers only settings and quit. blocked_options reports:

{"name":"timeline","enabled":false,
 "reason":"manual_epoch_reveal_required",
 "pending_epoch_ids":["NEOW_EPOCH"]}

menu_select("timeline") refuses:

"Timeline has obtained epochs that still need to be revealed manually;
 not opening Timeline because this game state logs invalid unlock-state
 errors when entered through automation"

This is a deliberate guard in the mod, not a bug. The user must open the Timeline in-game and reveal NEOW_EPOCH by hand. Until then the bot cannot start a new run.

Test suite

test_facts.py: 29 assertions, all passing, including the exact case where jev-1.13.0 answered "lethal available" 0.79 YES when the true answer was NO.