0xrsydn/sts2-bot
1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs(research): synthesis of game, state and primitives; next iteration

Browse source 
Join the three evidence sources that were kept apart, and extract the useful
result: each source independently decides what belongs in code and what belongs
to the model, and every serious bug we have found broke one of those rules.

  * Game mechanism  -> arithmetic and rules belong in CODE
  * Internal state  -> anything directly readable or writable belongs in CODE
  * TypeSafe        -> judgements go to the MODEL, in the primitive matching the
                       answer's shape

  * never blocked in boss fights        -> game mechanism (arithmetic left to the model)
  * card quality asked as a Noul        -> primitives (a spectrum forced into yes/no)
  * indices read from the wrong array   -> internal state

The counter-example is kept too: the Score we tried for fight-level planning was
unusable, so the rule is "match the primitive to the shape, and verify" rather
than "use Score more".

Also records the ordered plan (measurement first, then re-run, then primitives,
then deck composition), the continual-learning design, and why it is blocked:
with 0 wins in 37 runs the reward has no gradient, so training would fit the bug
rather than the game.
This commit is contained in:
0xrsydn 2026-09-22 06:06:21 +07:00
commit 65fe1171cd
2 changed files with 357 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

335
docs/research/10-synthesis-and-next-iteration.md Normal file
View file

@ -0,0 +1,335 @@
# 10 — Synthesis: game, state and primitives → next iteration
Status: **forward-looking**. This doc is the join of three evidence sources that
until now were kept apart:
| Source | Doc | What it gives us |
|---|---|---|
| STS2 game mechanism | `01`, `08` | What actually decides a run |
| Internal game state we control | `04`, `08` | What is observable, and what we can change |
| TypeSafe / System One primitives | `02`, `09` | What shape to ask the model |
It records the design conclusions and the planned iteration. Nothing here is a
measurement unless it says so; the measurements live in `05`, `07`, `08`, `09`.
---
## 1. The synthesis in one line
**Each source independently decides what belongs in code and what belongs to the
model — and every serious bug we have found is a violation of one of those three
rules.**
| Source | Rule it imposes |
|---|---|
| Game mechanism | Anything that is arithmetic or a rule belongs in **code**. HP, block, lethal, damage projection, energy. |
| Internal state | Anything we can read or write directly belongs in **code**. Index spaces, save files, prefs, action enumerations. |
| TypeSafe primitives | Anything that is a *judgement over unstructured meaning* goes to the **model** — and must use the primitive that matches the answer's shape. |
### The three bugs, mapped to the three rules
| Bug | Rule broken | Doc |
|---|---|---|
| Bot never blocked in boss fights; 9 runs lost to `THE_KIN_BOSS` | **Game mechanism.** Blocking is arithmetic (`projected_incoming > affordable_loss`) and we left it to a damage-biased model question. | `08` |
| Card quality asked as a `Noul` and `argmax`'d | **Primitives.** A spectrum judgment forced into yes/no. `primitives.md`: "A Noul value of 0.5 means the model gives yes and no equal probability. It does not mean the candidate has a medium skill level." | `09`, §2 |
| Card indices read from the wrong array (5 spaces, similar names) | **Internal state.** We misread what the index pointed at. | `04` |
That mapping is the useful part. It turns three unrelated bug hunts into one
checkable rule, and it predicts where the next bug is: **any place we ask the
model something that is really arithmetic, or use the wrong primitive for the
answer shape.**
### The counter-example that keeps us honest
`09` §3 records a `Score` we tried for fight-level planning ("Race / Trade /
Control") that was **unusable on 4 of 6 states**, confidence as low as 0.01.
So the rule is not "use Score more". It is "match the primitive to the answer
shape, and verify the shape works before adopting it."
---
## 2. What the game actually rewards (from `08`)
Established by mining 37 run files, not by reading guides:
- **81% of runs (30/37) die in Act 1.** Deck composition is downstream of that.
- **43% die to an Act 1 boss; `THE_KIN_BOSS` alone kills 9.**
- The bot is **not** bloating, descriptively: 23% of rewards were skipped (249 of
323 rewards took a card), one card per reward at most. The per-card rate
(249/1087 = 23%) is mechanically diluted by offer size and does not diagnose
skip policy; we have no STS2 benchmark for a good skip rate.
- The Kin fights lasted **5–10 turns** and cost **44–80 HP**, ~10–13 a turn,
with block cards in hand. That is a defense failure, and it is now fixed.
**Consequence for planning:** any work on archetypes, synergy scoring or deck
"lean weight" is tuning a system the bot does not survive long enough to use.
The order is: survive Act 1 → then optimise the deck.
---
## 3. What the state gives us that we are not yet using
Three levers found but not yet exploited:
### 3.1 The run files are a labelled dataset
`history/*.run` records, per map point: every card offered with `was_picked`,
`damage_taken`, `potion_used`, `rest_site_choices`, `upgraded_cards`, and the
final deck. That is a supervised dataset of our own drafting and combat, and it
already invalidated one hypothesis (bloat) and found one bug (defense).
**Not yet used for:** labelling decisions, or measuring the gates.
### 3.2 Instant Mode is a one-line lever
`prefs.save` holds `"fast_mode"` as plain JSON, and the mod's Instant Mode
checkbox is only `PrefsSave.FastMode = FastModeType.Instant`. Setting it to
`"instant"` removes animations and should roughly halve wall-clock per run.
**Needs a game restart** — prefs load at startup.
This matters because the binding cost of every experiment is wall-clock. At
~10 minutes per run, an A/B of 6v6 is two hours.
### 3.3 Model answers are now logged with the decision (not in a side trace)
**Corrected 2026-09-22.** This section previously said
`JEV_TRACE=capture/jev_trace.jsonl` was "the entire change". It is not, and it
was never sufficient on its own: `jev.py`'s trace records a `HH:MM:SS` stamp,
the questions and the answers — **no run, session, step, state or action**. Two
calls in the same second are indistinguishable, so the file cannot be joined to
a decision or to an outcome, which is the only thing a label needs.
What the decision log carries now, per decided action
(`run.py:decision_record`):
```json
{"session":"20260922T034419","step":55,"state_type":"monster",
"run":{"act":1,"floor":5},"action":"play_card","source":"jev",
"params":{"card_index":1,"target":"SHRINKER_BEETLE_0"},
"confidence":0.85,"reason":"jev chose Dismantle","error":null,
"jev":{"model":"jev-1.13.0","latency_s":0.71,
"questions":{...},"answers":{
"best_play":{"kind":"choice","choice":"card1","margin":0.79,"gated":true,
"probabilities":{"card0":0.01,"card1":0.89,...}},
"use_potion":{"kind":"noul","noul":0.41,"yes":false},
"which_potion":{"kind":"choice","choice":"potion0","margin":1.0,"gated":true}}}}
```
- `session` is a unique id per process (`<timestamp>-<6 hex>`). It is the join
key only because three places now carry it:
1. every decision row in `capture/decisions.jsonl`;
2. `capture/sessions.jsonl`, written by `run.py` at exit (via `atexit`, so it
covers every exit path) and mapping the session to the history run file(s)
it produced, with `win`, `killed_by`, `seed`, `deck_size`, `map_points`;
3. the A/B attribution row — `policy \t run \t code-hash \t session` — and the
report prints each arm's session ids.
The game's own run file does **not** know our session id and the harness
originally mapped only a policy to a filename, so without (2) and (3) nothing
lined up and there were still no labels.
- `jev` is `null` when the decision asked nothing (code paths, fallbacks), which
is a fact about the decision rather than a gap.
- `RecordingClient` in `run.py` captures the answers, so nothing in `brain.py`
had to change.
`JEV_TRACE` still works and is still worth setting, but it is now redundant for
labelling. It is no longer the prerequisite.
---
## 4. Where our Jev usage stands (from `09`, plus this audit)
Measured usage of the documented surface:
| Capability | Used |
|---|---|
| `Choice` | **7** questions — `best_play`, `which_potion`, `target`, `next_node`, `best_option`, `best_relic`, `best_bundle` |
| `Noul` | **6** question families — `use_potion`, `want_any`, `skip_all`, `good_card{i}`, `good_relic{i}`, `good_card{i}` (card_select), `worth_item{i}` |
| `Score` | **0** — though `score()` is defined at `jev.py:169` |
| Structured criteria (`entry()`) | **0** |
| Contrastive Noul criteria (`true=`/`false=`) | **0** |
| `focus=` / `inspect=` / `compare=` | **0** |
| `gate_choice()` / `margin()` | **0** in `brain.py` (`gate()` is used; `margin()` only inside `jev.py`) |
Every question is a bare one-line string. The docs permit that only when the
question is *short and unambiguous*; ours are neither, e.g.:
```python
noul("Is any relic in `relics` worth taking over skipping?")
```
`09` §2 measured structured criteria on card play: **same card chosen 6/6**,
margin 0.425 → 0.473. A small effect — worth adopting where disambiguation
costs us, not as a blanket rewrite. That measurement stands.
---
## 5. The next iteration, in dependency order
Ordered by dependency, not by appeal. Each step states what it unblocks.
### Step 1 — Enable measurement (blocks everything else)
**Partly landed 2026-09-22.** The answers and the join are in; the corpus is not.
Done:
- every decision row carries the model's questions and answers, or `null` when
nothing was asked (`run.py:decision_record`, `RecordingClient`);
- a unique `session` id on every row, and `capture/sessions.jsonl` at exit
mapping it to the run file(s) produced, with the outcome fields;
- `ab_card_skip.sh` stamps the same session into its attribution rows.
Still to do:
- capture **all** state types, not just combat (`run.py` currently dumps
`live_{step}_combat.json` only);
- namespace captures per run so they stop overwriting across sessions;
- log the *features* the decisions turn on (the `features` block below is a
sketch, not what is written today — the questions and answers are the
evidence currently recorded).
```json
{"run_id":"1790...","step":42,"state_type":"card_reward",
"features":{"card0_power":2.1,"card0_synergy":1.4},
"action":"select_card_reward:0","source":"jev","confidence":0.71,
"gate":"pass"}
```
**Why first:** it is the prerequisite for every quality question in `09` and for
continual learning (§6). With the join in place, the first usable dataset is one
A/B run away; what is missing now is coverage (all state types, namespaced per
run) and volume, not mechanism.
### Step 2 — Re-run and check whether the ceiling moved
The defense fix (`08`, failure #35) is the single highest-leverage change made
so far. Verify it with the A/B harness, now that `--stop-on-run-end` makes one
session equal one run:
```bash
./ab_card_skip.sh 6 # 6 runs per arm, one run per session
```
**The question:** does `THE_KIN_BOSS` stop killing 9 of 37? If yes, runs start
reaching Act 2+ and the reward signal finally varies. If no, the defense fix is
wrong and nothing downstream is worth building.
**Why second:** it is the test of the only substantive finding, and it gates §6.
### Step 3 — Match primitives to answer shapes
Only after Steps 1–2 give labels:
- Card quality: **`Score` with explicit ordered levels**, per card, several
atomic axes, combined with weights in code (composite scoring). Replaces the
`Noul` + `argmax` that the docs say is the wrong shape.
- Structured criteria on the fuzzy Nouls (`worth taking`, `stronger`).
- Measure each change against the Step 1 trace before keeping it.
### Step 4 — Then deck composition
Archetypes, synergy, removal priority. Last, because `08` shows the bot does not
currently reach the point where it matters.
---
## 6. Continual learning — the design, and the honest blocker
The docs give the mechanism directly:
> "Combine independent answers with deterministic rules or weighted sums. **For
> learned composition, use the probabilities as features in a downstream
> classical machine-learning model.**"
So continual learning here is **not fine-tuning Jev**. It is: Jev produces
features, code combines them, and **the weights are learned from outcomes**.
Four stages:
1. **Instrument** — Step 1 above.
2. **Label** — attach the run outcome to each decision row.
3. **Fit** — a small logistic regression or gradient-boosted tree on
`features → P(reach Act 2)`. The coefficients *are* the composite weights.
4. **Close the loop** — retrain periodically and A/B the new weights against the
old, using the `--stop-on-run-end` harness.
The docs also prescribe how to set our gates, which we have so far guessed:
> "Test thresholds by plotting confidence against accuracy on your data."
`EVENT_TOP_MIN = 0.60`, `EVENT_MARGIN_MIN = 0.30` and `CONFIDENCE_FLOOR = 0.55`
were chosen by hand. With a trace they can be **measured**. (Note that
`CONFIDENCE_FLOOR` is inert where it is passed: `gate()` ignores its threshold
argument for a `ChoiceAnswer`, so the target question is gated by the margin
rule, not by 0.55.)
### The blocker, stated plainly
**We have 0 wins in 37 runs.** The top of the reward scale is unobserved, so
there is nothing to learn from yet. Our only varying signal is map points
reached — noisy, and confounded by seed luck.
Training on this data would fit the bug, not the game. **Step 2 is what creates
the gradient.** If runs start reaching Act 2 and Act 3, a label appears
(`P(reach Act 2)`) that actually varies, and Stage 3 becomes possible.
---
## 7. Refactor implications
`brain.py` is ~1700 lines with 13+ handlers, and it now mixes four concerns:
state parsing, arithmetic, question building, and policy. The three-rule framing
in §1 suggests the split that the code keeps trying to make:
```
sts2bot/
game/ card and encounter knowledge, mechanics constants
state/ observation parsing, the five index spaces, CombatFacts
policy/ one module per state_type; chooses among enumerated actions
model/ TypeSafe primitives, a shared question library, the trace
learn/ decision log, labels, weight fitting
eval/ A/B harness, metrics, run-file mining
```
Two rules that should hold after the split:
1. **`policy/` may not do arithmetic.** If a decision needs a number, the number
comes from `state/`. This is the rule the defense bug broke.
2. **`model/` may not be asked a question whose answer is arithmetic.** This is
the rule the `Noul`-for-spectrum bug broke.
Both are testable as structural assertions, in the same style as the existing
`test_brain.py` hostile-index tests (indices from data, `OFFSET = 5`).
**Do not refactor yet.** The measurement in Step 1 should land first: it is
cheap, it is a prerequisite for everything, and moving files while the corpus
format is still changing would mean doing it twice.
---
## 8. Open questions carried forward
| Question | Blocked on |
|---|---|
| Do the gates (0.45 / 0.60 / 0.30) match their accuracy? | **per-decision labels** — replay, expert judgement, or a code-verifiable ground truth. Run outcomes cannot answer this: one run result attached to a step does not say whether that step's answer was correct. |
| Is structured criteria worth adopting beyond card play? | trace, then A/B |
| Does the two-stage cascade beat one broad question? | an optimal-action label |
| Does map beam search beat the single-step Choice? | `map_point_history` labels |
| Is `TREMBLE` (offered 45×, taken 0×) correctly rejected, or is that a bug? | a card-quality label |
| Does the event keyword veto reject benign options? | event captures + outcomes |
| Is the escalation tier (act / review / escalate) needed? | trace |
---
## 9. Summary
- **Three rules**, one per evidence source, and every serious bug broke one.
- **One fix landed** (defense) with the highest measured leverage: 9 runs, one
boss, one cause.
- **One blocker**: 0 wins means no reward gradient, so continual learning is
designed but cannot start.
- **One prerequisite**: enable `JEV_TRACE` and log decisions. Cheap, and it
unblocks every quality question above.
- **One test that matters next**: re-run and see whether `THE_KIN_BOSS` deaths
drop. That single number decides whether the rest of this plan is worth
building.

25
docs/research/README.md
View file

@ -18,6 +18,9 @@ guess, it says so.
| 05 | [Failure modes](05-failure-modes.md) | Every infinite loop found live, with its fix |
| 06 | [Decision architecture](06-decision-architecture.md) | The three-layer design and why the split is where it is |
| 07 | [Run log](07-run-log.md) | Results per run, with seeds and outcomes |
| 08 | [What actually wins](08-what-actually-wins.md) | Why the web meta is unusable, and the measured cause of 9 run losses |
| 09 | [TypeSafe best practice](09-typesafe-best-practice.md) | Doc-by-doc gap analysis: what we match, what measurement killed, what is missing |
| 10 | [Synthesis and next iteration](10-synthesis-and-next-iteration.md) | Game + state + primitives joined; the three rules; the planned iteration and refactor |
Architecture summary lives in [`../DESIGN.md`](../DESIGN.md).
@ -28,6 +31,15 @@ Architecture summary lives in [`../DESIGN.md`](../DESIGN.md).
3. Record the **date** and the **game build** (`v0.107.1` today). Both move.
4. When a finding is later disproved, do not delete it. Mark it superseded
and say what replaced it. The wrong turn is often the useful part.
5. **Re-measure numbers before you copy them.** Test counts, latencies and run
totals in these docs have gone stale more than once. The suites print their
own totals; use those, not a figure quoted from an earlier session.
> Worked example: a hand-off summary recorded `29` and `118` assertions.
> Re-running the suites showed `50` and `131`, and the difference was two
> regression sections that already existed in the files. The stale numbers
> would have gone into this index unchallenged. Run the suite; do not trust
> the summary.
## Environment these notes were taken on
@ -57,11 +69,12 @@ Then restart the game. Mods load only at process start.
## Testing
Two suites, both offline. Run them before every session.
Three suites, all offline. Run them before every session.
```bash
python3 test_brain.py # 54 assertions — structural / programmatic
python3 test_facts.py # 29 assertions — arithmetic and parsing
python3 test_brain.py # 131 assertions — structural / programmatic
python3 test_facts.py # 50 assertions — arithmetic and parsing
python3 test_run.py # 21 assertions — the decision log and its joins
```
`test_brain.py` is the important one for catching usage bugs. It asserts that
@ -69,6 +82,12 @@ every `state_type` produces an action **legal for that state**, that every
action the decision layer can emit is declared somewhere, and that every
fallback respects its own inputs. It needs no model and no running game.
`test_run.py` drives `main()` end to end with a fake `sts2` and a stub client,
so it pins the decision log without a network: a model decision's row carries
that decision's answers, and a decision that asked nothing logs `jev: null`
rather than inheriting the previous step's. Removing the reset before each
`decide()` makes it fail.
It was added after a session in which four separate infinite loops and three
hardcoded fallbacks were found by hand. Most of them would have been caught
here.