docs(research): record the defence failure mode and refresh the run log

Failure #35 is the largest defect found so far and it was invisible from the
code: 9 runs lost to one Act 1 boss, all with the same cause, all recorded in
the run files as damage taken, turns elapsed and potions spent. Recorded with
the reproduction, the fix, and the replay that verifies it.

Also adds a rule to the docs index: re-measure numbers before copying them.
Test counts, latencies and run totals in these notes have gone stale more than
once -- a hand-off summary recorded 29 and 118 assertions where the suites
actually printed 50 and 131.

docs/research/05-failure-modes.md

@@ -540,3 +540,110 @@ instead of waiting. Cost is one wasted action. No revalidation pass is needed.
 - `hand_select` confirms for `upgrade_select`
 - character select alternates select/embark and resets correctly
 - a `StubClient` makes the model paths testable offline and deterministically
+
+---
+
+## 35. Combat had no defense policy at all (the biggest one)
+
+Found by mining the 37 run files, not by reading the code.
+
+**Symptom.** 81% of runs (30/37) died in Act 1. `THE_KIN_BOSS` alone killed 9.
+Every one of those fights ran **5-10 turns** and cost **44-80 HP**, i.e. ~10-13 a
+turn, with block cards in hand the entire time.
+
+```
+damage taken: [44, 50, 50, 53, 63, 64, 70, 75, 80]
+turns:        [6, 9, 10, 6, 6, 10, 9, 7, 5]
+```
+
+**Cause A.** `facts.py` classes a hit of `<= 15% of max HP` as `THREAT_CHIP`.
+At 80 max HP that is **12**. `_fallback_combat` then required
+
+```python
+must_respect = threat in (HEAVY, SEVERE, LETHAL) or hp_bucket in (WOUNDED, CRITICAL)
+```
+
+so at 74/80 HP (HEALTHY) a 12-damage hit was **ignored** and the bot attacked.
+The boss's main attack sits exactly on that boundary. Measured:
+
+```
+  hp  incoming   threat  hp_bucket   old play
+  74        12     chip    healthy   Bash (dmg)
+  60        12     chip    healthy   Bash (dmg)
+  45        12     chip    wounded   Defend     <- only now, 29 HP already gone
+```
+
+**Cause B.** `_jev_combat` asked a `should_defend` Noul on **every combat turn
+and never read it**. `grep -rn should_defend *.py` returned exactly one line —
+the one that created it. Pure latency cost.
+
+**Cause C.** With that question dead, defense fell entirely to
+`choice("Which single play best advances winning this fight?")` — a phrasing
+biased to damage, in the `Choice` shape already measured as diluting with
+option count. Measured against the real Kin state, Jev answered **Bash at 0.42
+confidence**. 0.42 is below the 0.45 gate, so it fell through to the fallback,
+which also chose damage. **Both paths agreed on the wrong answer.**
+
+**Fix.** Blocking is arithmetic, so it is decided in code before Jev is asked:
+
+- `facts.CombatFacts.turns_to_kill` — remaining fight length from this turn's
+  reachable damage.
+- `facts.CombatFacts.projected_incoming` — `turns_to_kill * incoming_damage`.
+- `facts.CombatFacts.affordable_loss` — `hp - 30% of max_hp`.
+- `facts.CombatFacts.must_block` — `projected_incoming > affordable_loss`.
+- `combat_decision` now forces a block when `block_urgent` and a blocker is in
+  hand, **after** the lethal check, **before** Jev.
+- `should_defend` deleted.
+
+**Verification.** Replayed all 600 real combat captures. The rule changes
+**8 of 68** in-play turns (11.8%), and stays silent on short fights and when
+nothing is incoming:
+
+```
+live_162_combat.json    80/80  12 in   6 turns  proj 72 > afford 56  -> Defend
+live_238_combat.json    78/80  17 in  20 turns  proj 340 > afford 54 -> Rage
+trivial (15 HP enemy)                                           -> still attacks
+```
+
+Tests: 50 in `test_facts.py`, 131 in `test_brain.py` (the two lethal-search
+regressions in [09](09-typesafe-best-practice.md) §5 are included).
+
+**Lesson.** The bug was invisible from the code and obvious from the data. Nine
+runs died the same way, and the run files recorded the damage, the turn count
+and the potions spent. Mine the history before theorising about the meta.
+
+## 36. `relic_select` asked one question id and read another
+
+`relic_select_decision` builds its ranking questions keyed `good_relicN`:
+
+```python
+keys = [f"relic{r.get('index', 0)}" for r in relics]
+questions = {f"good_{k}": noul(...) for k in keys}
+```
+
+but then read the answers under the unprefixed ids:
+
+```python
+best_key, best_noul = best_by_noul(response, keys, CARD_PICK_THRESHOLD)
+```
+
+`best_by_noul` looks each key up with `response.get(key)`, so every lookup
+returned `None`, `ranked` was empty, and the function returned `(None, 0.0)`
+**for every state**. The relic path could therefore only ever take the rarest
+relic — Jev's answer was silently discarded on every boss and elite relic
+offer. The same shape is correct in `card_reward_decision` and
+`card_select_decision`, which keep the `good_` prefix end to end; relic_select
+was the one that did not.
+
+Nothing in the trace caught it, because no session in `decisions.jsonl` ever
+reached a `relic_select` state. Found by auditing every `best_by_noul` call
+site against the ids it asked for, not by a run.
+
+Fix: rank `[f"good_{k}" for k in keys]` and map the winner back with the same
+prefix. Regression test in `test_brain.py`: the highest-rated relic is
+deliberately the **Common** one, so the rarity fallback cannot produce the
+expected answer by accident — the test fails before the fix and passes after.
+
+**Lesson.** A question id that does not match its answer id fails silently and
+looks exactly like "the model had nothing to say". Every `best_by_noul` call
+site must be checked against the ids it actually asked for.

docs/research/07-run-log.md

@@ -204,3 +204,47 @@ went a full act further.
 The Timeline epoch reveal blocked the menu again after run 004, this time with
 `IRONCLAD2_EPOCH`. The mod refuses to automate it. A human must reveal it
 before the next run can start. Expect this after most runs.
+
+## Comparing two arms requires ONE revision
+
+A run is only comparable with another run collected from the **same revision of
+the decision layer**. `brain.py`, `facts.py`, `jev.py` and `run.py` decide how
+*both* arms behave, so a change to any of them — the fight-length projection,
+the lethal search, the state handed to Jev, the relic fix — invalidates every
+run collected before it. The old runs stay as history; they are not a baseline.
+
+`ab_card_skip.sh` now enforces this instead of relying on discipline:
+
+- it hashes the decision layer (`brain.py facts.py jev.py run.py sts2.py`) into
+  a 12-character content hash, printed as `code revision: <hash>`;
+- the hash is **recomputed before every session**, and the session line records
+  it: `--- jev 1/5 (attempt 1) rev=978485b6639a ---`. Taking it once at startup
+  would be worse than useless — `$RESULTS` is truncated when the run starts, so
+  every row would carry the startup hash by construction and the mixed-revision
+  warning could never fire;
+- a session whose hash differs from the startup hash **aborts the experiment**
+  (`!! ABORT: the decision layer changed mid-experiment`), skips the remaining
+  arm, and still prints the report for whatever was collected;
+- each attributed run is written as `policy \t run \t hash` in
+  `/tmp/ab_results.tsv`, stamped with the revision that actually produced it;
+- the report scores **one** revision — the one this invocation collected from —
+  and prints a `!! MIXED REVISIONS` block when the file holds more than one,
+  with a `!! THIN SAMPLE` warning under five runs per arm. It reads the same
+  history directory the shell snapshotted, instead of a second hardcoded copy of
+  that path (which silently reported "no runs recorded" for runs that existed).
+
+A content hash, not a commit id: the decision layer is normally edited in place
+and uncommitted, so a commit id would not tell two revisions apart.
+
+Verified in a sandbox with a stub `run.py` that edits `brain.py` between
+sessions: the abort fires, the remaining arm is skipped, and the collected rows
+keep the revision that produced them. A clean two-arm run stamps all four rows
+with one hash.
+
+**Rule:** before interpreting any A/B number, re-run BOTH arms from the same
+revision. Runs already in `/tmp/ab_results.tsv` from before 2026-09-22 03:2x
+are pre-change and must be re-collected.
+
+Note also that `--steps` must stay generous. A session that hits the step cap
+while the run is still going produces **no run record at all**, so a small step
+budget silently fills the sample with runs that died early.