72 lines
4 KiB
Markdown
72 lines
4 KiB
Markdown
# Experimental composite reward scoring
|
|
|
|
Enable with `python3 run.py --card-skip-policy composite --stop-on-run-end`.
|
|
This command makes model calls and executes game actions. Run it only when authorized.
|
|
The default remains `jev`. Shops, combat, and other selection screens are unchanged.
|
|
|
|
`policy/reward_scoring.py` builds context from observed reward metadata, player
|
|
resources, relics, and scoped combat-pile evidence. It preserves cost, rarity,
|
|
keywords, upgrades, and descriptions. It does not reconstruct the permanent deck.
|
|
|
|
One model call asks four independent Score questions per offered card:
|
|
|
|
| Dimension | Weight |
|
|
| --- | ---: |
|
|
| First-shuffle offense | 0.35 |
|
|
| Direct defense | 0.30 |
|
|
| Long-fight scaling | 0.15 |
|
|
| Draw, resource, and setup consistency | 0.20 |
|
|
|
|
Each dimension has five descriptive levels. Code normalizes each fractional score
|
|
with `(score - 2) / 2`, then computes the weighted sum. Rarity is context, not a
|
|
bonus. Pairing matters through observed support, not a separate synergy bonus.
|
|
The weights and positive-addition margin of 0.10 are experimental, not calibrated.
|
|
Utility is not a win probability or confidence.
|
|
|
|
## Module boundaries
|
|
|
|
- `policy/scoring.py` contains pure normalization, weighted utility, and stable
|
|
ranking functions. It has no Jev dependency, game rules, or acceptance thresholds.
|
|
- `policy/reward_scoring.py` owns reward context, rubrics, model calls, answer gates,
|
|
weights, and reward acceptance/fallback rules.
|
|
- `policy/selection.py` dispatches the selected reward policy.
|
|
- `run.py` executes decisions and records the scoring audit.
|
|
|
|
Shared arithmetic rejects missing components, invalid numbers, and invalid weights.
|
|
Weights must be nonnegative and sum to one. No component is silently removed.
|
|
Normalization maps each side of the neutral point linearly onto [-1, 1]. Ranking
|
|
preserves input order for ties. Callers retain responsibility for evidence quality.
|
|
Other policies can reuse these functions without sharing reward prompts or weights.
|
|
This extraction does not change reward behavior or resolve the skip limitation.
|
|
|
|
## Acceptance and limits
|
|
|
|
- Missing scoped card evidence or no model client uses the existing rarity fallback.
|
|
- Missing, nonfinite, out-of-range, or low-confidence components use that fallback.
|
|
- All four components for every card must pass. The current Score confidence gate
|
|
is 0.60. This is a heuristic gate, not proof of correct reasoning.
|
|
- Valid results select the highest weighted utility. Ties preserve offer order.
|
|
- A forced selection takes the highest utility even when all utilities are negative.
|
|
- **Automatic composite skipping is not implemented.** The serializer exposes
|
|
`can_skip`, but does not establish the alternative's effects. A weak or harmful
|
|
set therefore selects the highest-ranked card as a labeled fallback. This can
|
|
still add an unwanted card. Do not treat this version as a complete take/skip policy.
|
|
|
|
The permanent deck, card instance identity, and skip effects remain integration
|
|
prerequisites. Combat evidence can contain generated cards and temporary upgrades.
|
|
The model must not assume missing support or future upgrades. Better context does
|
|
not establish better decisions; compare recorded decisions before promoting this mode.
|
|
|
|
Proposal and compatibility records contain `scoring`: policy version, weights,
|
|
margin, evidence provenance, raw components, acceptance flags, utilities when
|
|
available, and the actual gate reason. Existing linked model records retain the
|
|
questions and parsed responses. No outcome becomes a per-decision correctness label.
|
|
|
|
Offline regression coverage is in `test_brain.py`. It checks batched questions,
|
|
fractional arithmetic, indices, context preservation, fallback paths, forced
|
|
selection, and recording. No live game or model connection is needed.
|
|
|
|
See [the research proposal](research/15-composite-card-policy-proposal.md) for the
|
|
full algorithm and evaluation plan. This implementation is its first, limited stage.
|
|
The [reward API audit](research/16-reward-api-audit.md) identifies the observation
|
|
changes needed before completing take-versus-skip decisions.
|