sts2-bot/docs/research/10-synthesis-and-next-iteration.md

10 — Synthesis: game, state and primitives → next iteration

Status: forward-looking. This doc is the join of three evidence sources that until now were kept apart:

Source	Doc	What it gives us
STS2 game mechanism	01, 08	What actually decides a run
Internal game state we control	04, 08	What is observable, and what we can change
TypeSafe / System One primitives	02, 09	What shape to ask the model

It records the design conclusions and the planned iteration. Nothing here is a measurement unless it says so; the measurements live in 05, 07, 08, 09.

---

1. The synthesis in one line

Each source independently decides what belongs in code and what belongs to the model — and every serious bug we have found is a violation of one of those three rules.

Source	Rule it imposes
Game mechanism	Anything that is arithmetic or a rule belongs in code. HP, block, lethal, damage projection, energy.
Internal state	Anything we can read or write directly belongs in code. Index spaces, save files, prefs, action enumerations.
TypeSafe primitives	Anything that is a judgement over unstructured meaning goes to the model — and must use the primitive that matches the answer's shape.

The three bugs, mapped to the three rules

Bug	Rule broken	Doc
Bot never blocked in boss fights; 9 runs lost to THE_KIN_BOSS	Game mechanism. Blocking is arithmetic (projected_incoming > affordable_loss) and we left it to a damage-biased model question.	08
Card quality asked as a Noul and argmax'd	Primitives. A spectrum judgment forced into yes/no. primitives.md: "A Noul value of 0.5 means the model gives yes and no equal probability. It does not mean the candidate has a medium skill level."	09, §2
Card indices read from the wrong array (5 spaces, similar names)	Internal state. We misread what the index pointed at.	04

That mapping is the useful part. It turns three unrelated bug hunts into one checkable rule, and it predicts where the next bug is: any place we ask the model something that is really arithmetic, or use the wrong primitive for the answer shape.

The counter-example that keeps us honest

09 §3 records a Score we tried for fight-level planning ("Race / Trade / Control") that was unusable on 4 of 6 states, confidence as low as 0.01. So the rule is not "use Score more". It is "match the primitive to the answer shape, and verify the shape works before adopting it."

---

2. What the game actually rewards (from 08)

Established by mining 37 run files, not by reading guides:

• 81% of runs (30/37) die in Act 1. Deck composition is downstream of that.
• 43% die to an Act 1 boss; THE_KIN_BOSS alone kills 9.
• The bot is not bloating, descriptively: 23% of rewards were skipped (249 of 323 rewards took a card), one card per reward at most. The per-card rate (249/1087 = 23%) is mechanically diluted by offer size and does not diagnose skip policy; we have no STS2 benchmark for a good skip rate.
• The Kin fights lasted 5–10 turns and cost 44–80 HP, ~10–13 a turn, with block cards in hand. That is a defense failure, and it is now fixed.

Consequence for planning: any work on archetypes, synergy scoring or deck "lean weight" is tuning a system the bot does not survive long enough to use. The order is: survive Act 1 → then optimise the deck.

---

3. What the state gives us that we are not yet using

Three levers found but not yet exploited:

3.1 The run files are a labelled dataset

history/*.run records, per map point: every card offered with was_picked, damage_taken, potion_used, rest_site_choices, upgraded_cards, and the final deck. That is a supervised dataset of our own drafting and combat, and it already invalidated one hypothesis (bloat) and found one bug (defense).

Not yet used for: labelling decisions, or measuring the gates.

3.2 Instant Mode is a one-line lever

prefs.save holds "fast_mode" as plain JSON, and the mod's Instant Mode checkbox is only PrefsSave.FastMode = FastModeType.Instant. Setting it to "instant" removes animations and should roughly halve wall-clock per run. Needs a game restart — prefs load at startup.

This matters because the binding cost of every experiment is wall-clock. At ~10 minutes per run, an A/B of 6v6 is two hours.

3.3 Model answers are now logged with the decision (not in a side trace)

Corrected 2026-09-22. This section previously said JEV_TRACE=capture/jev_trace.jsonl was "the entire change". It is not, and it was never sufficient on its own: jev.py's trace records a HH:MM:SS stamp, the questions and the answers — no run, session, step, state or action. Two calls in the same second are indistinguishable, so the file cannot be joined to a decision or to an outcome, which is the only thing a label needs.

What the decision log carries now, per decided action (run.py:decision_record):

{"session":"20260922T034419","step":55,"state_type":"monster",
 "run":{"act":1,"floor":5},"action":"play_card","source":"jev",
 "params":{"card_index":1,"target":"SHRINKER_BEETLE_0"},
 "confidence":0.85,"reason":"jev chose Dismantle","error":null,
 "jev":{"model":"jev-1.13.0","latency_s":0.71,
        "questions":{...},"answers":{
          "best_play":{"kind":"choice","choice":"card1","margin":0.79,"gated":true,
                       "probabilities":{"card0":0.01,"card1":0.89,...}},
          "use_potion":{"kind":"noul","noul":0.41,"yes":false},
          "which_potion":{"kind":"choice","choice":"potion0","margin":1.0,"gated":true}}}}

• session is a unique id per process (<timestamp>-<6 hex>). It is the join key only because three places now carry it:
  1. every decision row in capture/decisions.jsonl;
  2. capture/sessions.jsonl, written by run.py at exit (via atexit, so it covers every exit path) and mapping the session to the history run file(s) it produced, with win, killed_by, seed, deck_size, map_points;
  3. the A/B attribution row — policy \t run \t code-hash \t session — and the report prints each arm's session ids. The game's own run file does not know our session id and the harness originally mapped only a policy to a filename, so without (2) and (3) nothing lined up and there were still no labels.
• jev is null when the decision asked nothing (code paths, fallbacks), which is a fact about the decision rather than a gap.
• RecordingClient in run.py captures the answers, so nothing in brain.py had to change.

JEV_TRACE still works and is still worth setting, but it is now redundant for labelling. It is no longer the prerequisite.

---

4. Where our Jev usage stands (from 09, plus this audit)

Measured usage of the documented surface:

Capability	Used
Choice	7 questions — best_play, which_potion, target, next_node, best_option, best_relic, best_bundle
Noul	6 question families — use_potion, want_any, skip_all, good_card{i}, good_relic{i}, good_card{i} (card_select), worth_item{i}
Score	0 — though score() is defined at jev.py:169
Structured criteria (entry())	0
Contrastive Noul criteria (true=/false=)	0
focus= / inspect= / compare=	0
gate_choice() / margin()	0 in brain.py (gate() is used; margin() only inside jev.py)

Every question is a bare one-line string. The docs permit that only when the question is short and unambiguous; ours are neither, e.g.:

noul("Is any relic in `relics` worth taking over skipping?")

09 §2 measured structured criteria on card play: same card chosen 6/6, margin 0.425 → 0.473. A small effect — worth adopting where disambiguation costs us, not as a blanket rewrite. That measurement stands.

---

5. The next iteration, in dependency order

Ordered by dependency, not by appeal. Each step states what it unblocks.

Step 1 — Enable measurement (blocks everything else)

Partly landed 2026-09-22. The answers and the join are in; the corpus is not.

Done:

• every decision row carries the model's questions and answers, or null when nothing was asked (run.py:decision_record, RecordingClient);
• a unique session id on every row, and capture/sessions.jsonl at exit mapping it to the run file(s) produced, with the outcome fields;
• ab_card_skip.sh stamps the same session into its attribution rows.

Still to do:

• capture all state types, not just combat (run.py currently dumps live_{step}_combat.json only);
• namespace captures per run so they stop overwriting across sessions;
• log the features the decisions turn on (the features block below is a sketch, not what is written today — the questions and answers are the evidence currently recorded).

{"run_id":"1790...","step":42,"state_type":"card_reward",
 "features":{"card0_power":2.1,"card0_synergy":1.4},
 "action":"select_card_reward:0","source":"jev","confidence":0.71,
 "gate":"pass"}

Why first: it is the prerequisite for every quality question in 09 and for continual learning (§6). With the join in place, the first usable dataset is one A/B run away; what is missing now is coverage (all state types, namespaced per run) and volume, not mechanism.

Step 2 — Re-run and check whether the ceiling moved

The defense fix (08, failure #35) is the single highest-leverage change made so far. Verify it with the A/B harness, now that --stop-on-run-end makes one session equal one run:

./ab_card_skip.sh 6        # 6 runs per arm, one run per session

The question: does THE_KIN_BOSS stop killing 9 of 37? If yes, runs start reaching Act 2+ and the reward signal finally varies. If no, the defense fix is wrong and nothing downstream is worth building.

Why second: it is the test of the only substantive finding, and it gates §6.

Step 3 — Match primitives to answer shapes

Only after Steps 1–2 give labels:

• Card quality: Score with explicit ordered levels, per card, several atomic axes, combined with weights in code (composite scoring). Replaces the Noul + argmax that the docs say is the wrong shape.
• Structured criteria on the fuzzy Nouls (worth taking, stronger).
• Measure each change against the Step 1 trace before keeping it.

Step 4 — Then deck composition

Archetypes, synergy, removal priority. Last, because 08 shows the bot does not currently reach the point where it matters.

---

6. Continual learning — the design, and the honest blocker

The docs give the mechanism directly:

"Combine independent answers with deterministic rules or weighted sums. For learned composition, use the probabilities as features in a downstream classical machine-learning model."

So continual learning here is not fine-tuning Jev. It is: Jev produces features, code combines them, and the weights are learned from outcomes.

Four stages:

1. Instrument — Step 1 above.
2. Label — attach the run outcome to each decision row.
3. Fit — a small logistic regression or gradient-boosted tree on features → P(reach Act 2). The coefficients are the composite weights.
4. Close the loop — retrain periodically and A/B the new weights against the old, using the --stop-on-run-end harness.

The docs also prescribe how to set our gates, which we have so far guessed:

"Test thresholds by plotting confidence against accuracy on your data."

EVENT_TOP_MIN = 0.60, EVENT_MARGIN_MIN = 0.30 and CONFIDENCE_FLOOR = 0.55 were chosen by hand. With a trace they can be measured. (Note that CONFIDENCE_FLOOR is inert where it is passed: gate() ignores its threshold argument for a ChoiceAnswer, so the target question is gated by the margin rule, not by 0.55.)

The blocker, stated plainly

We have 0 wins in 37 runs. The top of the reward scale is unobserved, so there is nothing to learn from yet. Our only varying signal is map points reached — noisy, and confounded by seed luck.

Training on this data would fit the bug, not the game. Step 2 is what creates the gradient. If runs start reaching Act 2 and Act 3, a label appears (P(reach Act 2)) that actually varies, and Stage 3 becomes possible.

---

7. Refactor implications

brain.py is ~1700 lines with 13+ handlers, and it now mixes four concerns: state parsing, arithmetic, question building, and policy. The three-rule framing in §1 suggests the split that the code keeps trying to make:

sts2bot/
  game/     card and encounter knowledge, mechanics constants
  state/    observation parsing, the five index spaces, CombatFacts
  policy/   one module per state_type; chooses among enumerated actions
  model/    TypeSafe primitives, a shared question library, the trace
  learn/    decision log, labels, weight fitting
  eval/     A/B harness, metrics, run-file mining

Two rules that should hold after the split:

1. policy/ may not do arithmetic. If a decision needs a number, the number comes from state/. This is the rule the defense bug broke.
2. model/ may not be asked a question whose answer is arithmetic. This is the rule the Noul-for-spectrum bug broke.

Both are testable as structural assertions, in the same style as the existing test_brain.py hostile-index tests (indices from data, OFFSET = 5).

Do not refactor yet. The measurement in Step 1 should land first: it is cheap, it is a prerequisite for everything, and moving files while the corpus format is still changing would mean doing it twice.

---

8. Open questions carried forward

Question	Blocked on
Do the gates (0.45 / 0.60 / 0.30) match their accuracy?	per-decision labels — replay, expert judgement, or a code-verifiable ground truth. Run outcomes cannot answer this: one run result attached to a step does not say whether that step's answer was correct.
Is structured criteria worth adopting beyond card play?	trace, then A/B
Does the two-stage cascade beat one broad question?	an optimal-action label
Does map beam search beat the single-step Choice?	map_point_history labels
Is TREMBLE (offered 45×, taken 0×) correctly rejected, or is that a bug?	a card-quality label
Does the event keyword veto reject benign options?	event captures + outcomes
Is the escalation tier (act / review / escalate) needed?	trace

---

9. Summary

• Three rules, one per evidence source, and every serious bug broke one.
• One fix landed (defense) with the highest measured leverage: 9 runs, one boss, one cause.
• One blocker: 0 wins means no reward gradient, so continual learning is designed but cannot start.
• One prerequisite: enable JEV_TRACE and log decisions. Cheap, and it unblocks every quality question above.
• One test that matters next: re-run and see whether THE_KIN_BOSS deaths drop. That single number decides whether the rest of this plan is worth building.