docs(research): propose evidence-based composite card scoring

docs/research/15-composite-card-policy-proposal.md

@@ -0,0 +1,291 @@
+# Proposal: evidence-based card selection with composite scoring
+
+Status: research and proposal only. No policy changes or approved architecture decision.
+
+## Recommendation
+
+Use code to build legal candidates and verified facts. Ask Jev for narrow judgments with descriptive Score levels. Combine those judgments in code, relative to the actual alternative. Route uncertain decisions to review or a documented fallback.
+
+Implement reward drafting first. Use a separate evaluator for combat actions and selection operations. A good reward card is not necessarily the best card to play now.
+
+Do not start with a large simulator, five duplicated character policies, or an untested universal card tier list.
+
+## Evidence and limits
+
+This proposal combines current Python code, the vendored STS2MCP serializer, official TypeSafe documentation, and STS2 community strategy guides.
+
+The local game API refused a read-only wiki request during this research. Installed-build card effects were therefore not reverified through that endpoint. The vendored serializer is evidence about source behavior, not proof of the loaded mod version.
+
+STS2 is in Early Access. Pin game, mod, model, and policy versions during evaluation. Community recommendations are strategy hypotheses, not executable rules or measured bot performance.
+
+### Sources
+
+1. [TypeSafe: State](https://docs.typesafe.ai/concepts/state) — structured facts; questions evaluate the same state independently.
+2. [Choice: batch questions](https://docs.typesafe.ai/primitives/choice#good-practice-ask-more-than-one-question-per-call) — parallel questions, low incremental latency, extra token cost.
+3. [Score](https://docs.typesafe.ai/primitives/score) — ordered descriptive levels, fractional score, distribution, and confidence.
+4. [Composite scoring](https://docs.typesafe.ai/patterns/composite-scoring) — independent dimensions combined with code-owned weights.
+5. [Confidence](https://docs.typesafe.ai/confidence) — distribution concentration is not correctness; thresholds require domain evaluation.
+6. [Re-ranking cookbook](https://docs.typesafe.ai/cookbooks/rerank_typesafe) — Noul probabilities can legitimately rank candidates against a defined criterion.
+7. [Skill suggestion cookbook](https://docs.typesafe.ai/cookbooks/skill_suggestion) — rank, then inspect shortlisted candidates with fuller evidence; allow no selection.
+8. [Mobalytics: Ironclad](https://mobalytics.gg/slay-the-spire-2/characters/ironclad-guide) — Japanese Export's work-in-progress guide emphasizes draw, energy, and Exhaust interactions, not only immediate damage.
+9. [Mobalytics: card tier methodology](https://mobalytics.gg/slay-the-spire-2/tier-lists/cards) — rankings are preliminary and do not determine a card's strength in every run.
+10. [Card rewards guide](https://www.spirebuilds.com/guides/understanding-card-rewards) — cards should solve current deck problems. Do not adopt its fixed deck-size cutoff as a rule.
+11. [Character mechanics research](14-character-mechanics-and-policy.md) — character-specific resources and timing, with developer references.
+
+## 1. What the bot actually knows
+
+### Available in the vendored serializer
+
+`McpMod.StateBuilder.cs` exposes:
+
+- Offered cards: ID, name, type, energy cost, Star cost, text, rarity, upgrade flag, keywords, and action index.
+- Player: character, HP, block, gold, status effects, relic text/counters, and potions.
+- Combat: hand legality, energy, Stars, pile contents, ordered orbs with passive/Evoke values, orb slots, and pets such as Osty.
+- Enemies: identities, HP, block, powers, and current intents.
+- Selection screens: prompt, mode, confirm availability, and some preview/selection evidence.
+
+Important limits:
+
+- `BuildPlayerState` does not serialize the permanent deck, despite an older nearby comment mentioning a deck.
+- Non-hand pile entries omit stable card-instance identity and some metadata.
+- The draw pile is sorted for display. Its list order is not draw order.
+- Orb values already incorporate Focus in the serializer. Do not apply Focus again.
+- Hand damage already incorporates attacker modifiers. Do not apply Strength or Weak again.
+- Displayed cost or upgrade state can be temporary. It is not necessarily the permanent card definition.
+- `can_skip` is inferred from alternative reward buttons. Verify what the actual alternative does before assigning it zero value.
+
+See `docs/RUN_STATE.md` for identity and evidence freshness. `RunContext` carries combat-pile counts only within a reported run and room. Generated cards, temporary upgrades, and missing piles prevent treating these counts as the permanent deck.
+
+### Information dropped by current policy
+
+`card_reward_decision` sends offered names, types, and text, but not explicit costs, relics, or keywords. Deck evidence is a name-count map rather than full card mechanics.
+
+`CombatFacts.to_state` has no explicit character resources, orb queue, or Osty state. Adding a better prompt cannot recover discarded fields.
+
+### Current decision weaknesses
+
+- Rewards: one broad Noul per card; a separate gated skip question; otherwise take the highest rating, even below 0.5.
+- Upgrade/remove/transform grids: broad per-card Nouls with a common threshold and heuristics.
+- Hand selection: does not call Jev; uses the same basic-card disposal rule across selection purposes. It can select a Strike before an upgrade preview, rather than evaluate upgrade benefit.
+- Combat: chooses card and target separately; may combine the best card with a target judged for a different attack.
+- Combat: no model-selectable `end_turn` while cards remain playable.
+- Combat: deterministic kill and defense rules run before Jev. Better model questions alone cannot repair mistakes in those rules.
+
+A Noul is not inherently a bad ranking primitive. TypeSafe explicitly demonstrates it. The problem here is an underspecified criterion, incomplete evidence, and inconsistent take/skip logic. Composite scoring is a testable alternative, not a guaranteed improvement.
+
+## 2. STS2 principles the algorithm must represent
+
+### Marginal value, not card reputation
+
+Evaluate the deck after a change against the deck without that change. Strong cards can be redundant or unaffordable. Ordinary cards can fix an immediate weakness.
+
+A larger deck is not automatically worse. Draw, Exhaust, generated cards, required coverage, and encounter demands change the value of an additional card. Do not impose a universal deck-size cap or fixed number of attacks.
+
+### Multiple time horizons
+
+Early damage can prevent repeated attacks. Defense can preserve enough HP to reach the next recovery opportunity. Scaling matters when a fight lasts long enough to use it.
+
+Use observed route and boss information where available. Do not invent the next encounter or let Jev assume an unseen map. A stage label is a prior, not a substitute for actual deck needs.
+
+### Functional support, not an archetype label
+
+A payoff needs usable support. Count or evaluate actual energy, draw, block, discard, resource generation, and triggers. Do not add a generic synergy bonus for sharing a keyword.
+
+For example, Body Slam can use block gained on the same turn. Barricade is not a mandatory prerequisite. Conversely, merely owning Barricade does not establish affordable setup or sufficient block generation.
+
+### Execution matters
+
+A deck evaluator must consider whether the combat policy can use the chosen engine. Missing mechanic support should produce an explicit support warning. It should not silently rewrite the game's card quality.
+
+Ironclad needs HP-spending and Exhaust sequencing; Silent needs discard/Sly and delayed Poison; Defect needs orb order; Regent needs two resources; Necrobinder needs Doom timing and companion state.
+
+## 3. Reward algorithm
+
+### Step A: build a truthful decision state
+
+Keep these sections separate:
+
+- `observed`: run identity, location, offered card fields, HP, relics, potions, and known route.
+- `card_evidence`: permanent deck if verified; otherwise labeled combat evidence with source and freshness.
+- `computed`: supported counts, resource comparisons, and capability gaps.
+- `unknown`: missing deck identity, unsupported interactions, unknown upgrades, and unavailable encounter facts.
+- `candidates`: take each offered card and the actual skip/alternative action when legal.
+
+A verified permanent-deck source is the preferred prerequisite for confident deck-level judgments. A narrow read-only mod addition is a candidate future task, not part of this proposal's implementation permission. Do not read arbitrary save paths or infer an exact deck from pile unions.
+
+### Step B: ask four outcome questions per card
+
+Evaluate the change relative to declining that card, using only supplied evidence:
+
+| Dimension | Question scope | Exclude |
+| --- | --- | --- |
+| Early offense | Change in the deck's ability to remove threatening enemies during the first shuffle | Long-fight growth already scored below |
+| Defense | Change in reliable damage prevention or recovery during the first shuffle | Damage prevented by kills already credited to offense |
+| Long-fight scaling | Change in sustainable performance over repeated turns/shuffles | Immediate damage alone |
+| Execution consistency | Change in access to useful plays, resource fit, draw, and setup reliability | Another bonus for the same raw attack/block output |
+
+Use one consistent five-level direction: harmful, neutral, beneficial. Define each level with concrete situations for that dimension. Do not use bare numeric scales or adjectives alone.
+
+Example rubric for execution consistency:
+
+0. Adds a recurring unusable draw or requires a resource the supplied deck cannot provide.
+1. Makes useful turns less reliable through redundant cost or unsupported setup.
+2. No supported material change to useful-play access or setup reliability.
+3. Relieves an identified draw, cost, or setup bottleneck using support already present.
+4. Resolves a major identified bottleneck with reliable supplied support.
+
+Unknown is not level 2. If required evidence is absent, mark the dimension unavailable before scoring or route the decision to review.
+
+For a standard three-card reward, this is 12 Score questions in one call. An optional narrow evidence-sufficiency question per card makes 15. Code still validates evidence presence; a model's sufficiency answer cannot prove it.
+
+All questions in one request are independent. A score question cannot consume a gap diagnosis returned by another question in that same request. Compute needs first, or use a separately measured second request.
+
+### Step C: compose in code
+
+For Score levels 0 through 4:
+
+```text
+marginal_dimension = (returned_score - 2) / 2
+utility(card) = sum(weight[dimension] * marginal_dimension)
+```
+
+This maps the declared neutral level to zero. It is an explicit policy utility scale, not expected win probability or expected HP. Ordinal levels are not naturally equal-interval measurements; equal spacing is a starting modeling choice that needs validation.
+
+Illustrative starting weights, not tuned defaults:
+
+| Context | Early offense | Defense | Scaling | Consistency |
+| --- | ---: | ---: | ---: | ---: |
+| Early run with weak immediate damage | .35 | .30 | .15 | .20 |
+| Supported long-fight concern | .20 | .30 | .30 | .20 |
+
+Set the profile from verified context and documented rules. If a deck weakness is model-inferred, record it as an estimate, not a fact. Begin with one fixed profile to establish whether decomposition itself helps, then test contextual weights separately.
+
+Avoid a separate synergy bonus: synergy should change the relevant outcome scores. Otherwise the same interaction can receive credit twice. The four dimensions still interact; independence of API evaluation is not statistical independence.
+
+A plain skip that leaves the deck unchanged has zero marginal utility by definition. Relic effects or other alternatives can change that baseline and must be modeled. If those effects are unknown, do not assert that skip is free.
+
+### Step D: apply acceptance and uncertainty rules
+
+1. Remove only illegal candidates or candidates excluded by verified constraints. Missing support is not proof of illegality.
+2. Compute utilities and retain the per-dimension distributions.
+3. Compare the best candidate with the actual alternative, not only other cards.
+4. Require a validated positive margin before an optional card addition.
+5. Route missing evidence, near ties, and unstable rankings to review or a documented fallback.
+
+Do not use `utility * confidence`. An uncertain negative score would become less negative and could incorrectly improve a candidate's rank.
+
+Do not multiply Noul answers into a purported joint success probability. The questions share state and model biases. Likewise, summed per-axis variances do not establish an uncertainty interval without covariance assumptions and calibration.
+
+Use sensitivity checks instead: does a plausible one-level change on an uncertain dimension reverse the winner or its take/skip decision? This is a review heuristic, not a calibrated probability.
+
+Skipping is not a universally safe fallback, especially for a weak starter deck. Unknown deck evidence should trigger a state-limited fallback or review, not silent optimism or automatic skipping.
+
+### Step E: optional second-stage comparison
+
+Only for ambiguous, high-impact rewards, send the top two candidates plus skip in a second call. Include their actual evidence and unresolved trade-off. Ask a Choice with an explicit review/insufficient-evidence option.
+
+Avoid using a second call as ritual self-confirmation. It provides no independent correctness guarantee. It must receive useful additional context or a narrower comparison, and earn its cost in evaluation.
+
+## 4. Other card selection operations
+
+Use the same state discipline, but different candidate changes:
+
+- Add: evaluate `deck + card` against the legal alternative.
+- Remove: evaluate `deck - card`; account for lost coverage, not only deck size.
+- Upgrade: evaluate the actual upgraded effect against the current effect. Do not assume all upgrades are numeric increases.
+- Transform: evaluate a distribution only if the pool and rules are known. Never invent the resulting card.
+- Discard/Exhaust: evaluate immediate resource and trigger effects plus future availability. These operations are not interchangeable.
+- Multi-select: evaluate combinations or condition later choices on accepted selections. Independent top-N rankings can remove all defense or double-count a synergy.
+- Confirm: use deterministic screen rules and the pending-action guard. Confirmation does not need a model call.
+
+A small operation classifier in code should distinguish upgrade, discard, Exhaust, replace, and mandatory versus optional selection. Ambiguous prompts should not default to destroying a useful card.
+
+## 5. Combat algorithm: a separate short-horizon planner
+
+Do not reuse reward weights for combat.
+
+### Candidate actions
+
+Construct legal action-target pairs, not independent card and target choices. Include relevant potions and `end_turn`. Use game legality fields and verified resource facts.
+
+Retain one-action execution followed by a fresh observation. Planned sequences are proposals, not a queued list of game actions.
+
+### Deterministic evaluation first
+
+For supported mechanics, evaluate damage, block, resource use, retaliation, and lethal timing in code. Distinguish a local enemy kill from winning the encounter or surviving the turn.
+
+A simple direct-damage kill should not automatically outrank a survival-preserving alternative in multi-enemy combat. Estimated fight length should not become an unconditional defense veto.
+
+When rules are unsupported, keep results unknown. Never treat a partial simulator as proof that an action is safe or lethal.
+
+### Bounded search plus semantic evaluation
+
+Start with shallow search over deterministic actions. A small beam, such as width 4 and depth 2, is an experimental cost bound, not a tuned optimum. End a branch at unknown draws, random targets, hidden effects, or selection screens unless that effect is modeled.
+
+Preserve survival candidates, resource generators, setup actions, and end turn when pruning. Immediate-damage-only pruning would discard the combinations the search is intended to find.
+
+Ask Jev only about residual qualitative trade-offs: setup usefulness, future engine value, and threat priority. Do not ask it to recompute arithmetic already supplied by the planner.
+
+Rank supported continuations by survival constraints first, then expected combat value. Reobserve after executing only the first action. Random outcomes need explicit chance handling; sorted pile display must never be used as a draw oracle.
+
+This is a later phase. Reward scoring and truthful state should be completed before adding a broad combat simulator.
+
+## 6. Validation before claiming improvement
+
+### Recording prerequisite
+
+The linked recorder now preserves full observations and successful model request states; see [recording documentation](../RECORDING.md). Older reward rows alone cannot reliably reconstruct rejected alternatives. Composite-score components, weights, and actual policy-gate reasons still need explicit records.
+
+Record observation/request linkage, candidate IDs and indices, card definitions, evidence provenance, actual model version, question schema version, weights, component scores, distributions, gate reasons, and action results. Exclude credentials.
+
+### Offline correctness
+
+Use model stubs for ordinary tests. Cover:
+
+- All cards harmful versus skip; all cards neutral; one clear improvement.
+- Low probability of benefit is not automatically utility magnitude.
+- No positive minimum acceptance bug.
+- Missing deck or temporary cards cannot establish permanent synergy.
+- High-cost and Star-cost requirements remain explicit.
+- Real upgrade deltas and operation-specific hand selections.
+- No fabricated transform outcome or draw order.
+- Confidence does not alter the sign of a negative utility.
+- Multi-enemy kill is not an encounter-win proof.
+- End turn remains a candidate when legal.
+- Rejected/accepted requests and delayed confirmation preserve current invariants.
+
+### Human-reviewed decision set
+
+Start with a diverse set of roughly 50–100 reward states, including early picks, redundant cards, existing engines, expensive setup, missing evidence, and different characters when supported. This is a development set, not sufficient proof of a win-rate improvement.
+
+Label acceptable choices and rationale, allowing several defensible answers. Split by run. Fit rubrics and thresholds on development runs, then hold them fixed on separate test runs.
+
+Compare these ablations:
+
+1. Current policy.
+2. Better state with the current questions.
+3. Better state plus explicit Choice over cards and skip.
+4. Better state plus composite scoring.
+5. Composite scoring plus an ambiguity-only second call.
+
+This distinguishes improvements from extra context, extra calls, and the scoring method itself.
+
+### Authorized live evaluation
+
+Keep character, ascension, game/mod/model version, execution speed, and combat policy fixed when evaluating reward drafting. Report boss reach, wins, HP loss, invalid actions, fallback rate, latency, and tokens/cost.
+
+Use runs as the evaluation unit. Do not label every reward in a losing run wrong. Paired seeds can reduce variance, but different choices can alter later random events, so they are not identical counterfactuals.
+
+No new model calls, evaluation batches, game actions, code changes, or vendor edits were made for this proposal. The attempted local wiki GET was read-only and failed.
+
+## 7. Suggested delivery order
+
+1. Use the linked recording foundation to build reward replay and evaluation inputs.
+2. Obtain verified permanent-deck data, or explicitly limit the first evaluator to available evidence.
+3. Add operation-aware selection and the missing skip/end-turn alternatives where applicable.
+4. Add opt-in composite reward scoring with stubs, descriptive rubrics, and transparent weights.
+5. Review disagreements against the simpler baselines before enabling it live.
+6. Add correct upgrade/remove/hand-selection evaluation.
+7. Add character-aware combat facts and bounded planning as a separate change.
+
+The first milestone should be an auditable reward evaluator, not a claim of solved drafting or a complete game AI.