fix(policy): reconcile session-owned action memory with game observations
This commit is contained in:
parent
3f243eaeee
commit
bf41945ef9
9 changed files with 405 additions and 362 deletions
|
|
@ -117,7 +117,8 @@ The audit remains diagnostic, not a pass/fail suite. It also reports known issue
|
|||
|
||||
## Next: policy state and recording
|
||||
|
||||
Still open:
|
||||
Update: [the policy-state pass](13-policy-state.md) addresses item 1 below.
|
||||
The list records the remaining work at the end of this correctness pass:
|
||||
|
||||
1. Selection/shop/minigame memory can change when an action is proposed, before execution succeeds.
|
||||
2. Captures cover only combat and reuse filenames across sessions.
|
||||
|
|
|
|||
113
docs/research/13-policy-state.md
Normal file
113
docs/research/13-policy-state.md
Normal file
|
|
@ -0,0 +1,113 @@
|
|||
# Session-owned policy state
|
||||
|
||||
## Scope
|
||||
|
||||
This pass replaces per-screen globals with explicit session memory. It follows
|
||||
[the correctness pass](12-correctness-pass.md). It does not redesign recording,
|
||||
identify runs, or establish persistent-deck provenance.
|
||||
|
||||
`brain.py` remains one file. Moving policy code and changing its state behavior
|
||||
at the same time would make failures harder to diagnose. A later extraction
|
||||
can preserve the interface and the runner-level tests.
|
||||
|
||||
## Lifecycle
|
||||
|
||||
1. `run.main()` creates one `brain.PolicyContext` for the session.
|
||||
2. `brain.decide(..., context=context)` reconciles the fresh observation.
|
||||
3. The policy returns a `Decision`, which is only a proposal.
|
||||
4. The runner executes the action unless this is a dry-run or a suppressed duplicate.
|
||||
5. The runner calls `context.record_result(..., accepted=result.ok)`.
|
||||
6. The next observation supplies evidence before further guarded actions.
|
||||
|
||||
A rejected action does not reserve a card index, consume a crystal cell, or
|
||||
mark a reward as skipped. A rejected embark clears the accepted character
|
||||
selection, so the next proposal selects the character again.
|
||||
|
||||
Transport errors stop the runner. They can leave the game's action outcome
|
||||
unknown. The runner does not retry those POSTs automatically.
|
||||
|
||||
Calling `decide` without a context performs independent snapshot analysis.
|
||||
Sequential callers must retain a context and report real action results.
|
||||
Direct screen handlers are policy helpers, not a replacement for this lifecycle.
|
||||
|
||||
## Evidence and limits
|
||||
|
||||
An accepted response does not prove completion. `PendingAction` represents an
|
||||
accepted request awaiting reconciliation, not a successful game effect.
|
||||
|
||||
| Action | Condition for further progress |
|
||||
| --- | --- |
|
||||
| Grid toggle | A fresh read of the same grid reserves the accepted index. Selection itself remains unverified. |
|
||||
| Direct choose-card screen | A screen-flow transition. |
|
||||
| Hand selection | The observed selected-card count increases, or the screen changes. Selected-list indices use a separate index space. |
|
||||
| Grid, hand, or bundle confirmation/cancellation | A screen-flow transition. No automatic repeated confirm or speculative cancel. |
|
||||
| Bundle selection | An observable preview with confirmation enabled, or a screen transition. |
|
||||
| Purchase | The targeted inventory item changes or disappears, or the screen changes. Price and affordability changes alone do not release the guard. |
|
||||
| Crystal click | The cell is no longer clickable, progression unlocks, or the screen changes. |
|
||||
| Character selection | A fresh read releases the accepted request. The mod does not expose the selected character. |
|
||||
| Embark | A screen transition. Rejection instead allows re-selection. |
|
||||
| Card-reward skip | The reward list reappears before the policy marks the reward as skipped. |
|
||||
|
||||
### Missing selection information
|
||||
|
||||
The vendored mod's `BuildCardSelectState` exposes grid cards, preview state,
|
||||
and confirmation availability. It does not expose all selected card indices.
|
||||
`ExecuteSelectCard` reports that it emitted a toggle signal, not that the
|
||||
selection is complete.
|
||||
|
||||
The policy therefore stores **accepted toggles**, not confirmed selections.
|
||||
This permits multi-select screens whose observation remains unchanged between
|
||||
selections. It excludes accepted indices from later proposals in the same flow.
|
||||
If the grid changes while accepted indices remain in use, the policy waits
|
||||
rather than mapping old indices onto the changed grid.
|
||||
|
||||
This is a limited inference, not a full game-state model. A lost accepted toggle
|
||||
can stall the flow. Restarting the bot on a partially selected grid also loses
|
||||
the acceptance ledger. Reliable recovery needs selected-card identities from
|
||||
the mod, or an explicit operator reset. This pass adds neither.
|
||||
|
||||
### Screen boundaries and waiting
|
||||
|
||||
- `rewards` and `card_reward` share one flow, so skipping survives the return to rewards.
|
||||
- Menu screen names distinguish character selection from the main menu.
|
||||
- `unknown` and `overlay` do not reset memory or prove completion.
|
||||
- Other observed flow changes clear screen memory and pending requests.
|
||||
- An unchanged screen does not prove an action failed. The policy waits instead of cancelling a slow confirmation.
|
||||
- The existing stuck limit bounds unchanged observations. A separate pending-action deadline uses the same `--stuck-seconds` limit, even when unrelated fields change.
|
||||
- A bounded session can still end at its step limit with an unresolved action. Exit zero for that mode does not claim action completion.
|
||||
|
||||
Same-type screen replacements are not uniquely identified. There is no run or
|
||||
screen-instance identifier in the context yet. Recovery is conservative: an
|
||||
ambiguous flow can stop rather than risk another purchase or toggle.
|
||||
|
||||
## Validation
|
||||
|
||||
No live game connection, paid model calls, new dependencies, or new test files.
|
||||
|
||||
- Existing scripts: 68 facts, 110 policy, and 84 runner assertions passed; 262 total.
|
||||
- Runner flows cover rejected and delayed selections, confirmation, purchases,
|
||||
bundles, crystal clicks, character selection, reward skipping, session
|
||||
isolation, dry-run, and pending-action timeout under unrelated state changes.
|
||||
- Old tests that advanced policy memory by merely proposing actions were removed.
|
||||
The combined policy/runner test code is smaller than before this pass.
|
||||
- Thirteen temporary whole-process scenarios passed with real HTTP clients and
|
||||
loopback fixture servers. These include the eight prior correctness scenarios
|
||||
and five selection, dry-run, purchase, bundle, and crystal flows.
|
||||
- The offline audit replayed 346 stored observations without exceptions.
|
||||
Repeated proposals with one retained context no longer consume a selection.
|
||||
- Dataset integrity: 37 runs, 1,052 decisions, and 346 observations passed.
|
||||
- Shell syntax checks passed.
|
||||
|
||||
Temporary probes are not part of the repository. The permanent state regressions
|
||||
use the actual runner and policy together, with isolated game/model boundaries.
|
||||
|
||||
## Next boundaries
|
||||
|
||||
The state interface now gives a stable boundary for extracting combat and
|
||||
selection policy without changing execution behavior. Keep a small dispatcher;
|
||||
no plugin framework or class hierarchy is needed.
|
||||
|
||||
The next state work is run identity and deck provenance. Recording should then
|
||||
link observations, proposals, action attempts, results, and reconciliation.
|
||||
Session finalization, capture-name collisions, and actual policy-gate metadata
|
||||
remain separate work.
|
||||
Loading…
Add table
Add a link
Reference in a new issue