Commit graph

2 commits

Author SHA1 Message Date
0xrsydn
d676bf067e fix(eval): end each A/B session with its run, and attribute runs to arms
Two defects made the earlier A/B numbers meaningless:

  * Without `--stop-on-run-end` a session contained several runs, so runs could
    not be attributed to an arm.
  * The script then read "the last 2N runs", which silently compared runs from
    earlier experiments rather than the two arms.

Pass `--stop-on-run-end`, and attribute each run record to its arm by diffing
the history directory around each session.

Also raise the step budget and warn on a thin sample: measured, a run still
going at step 600 produces NO run record, so a short budget biases the sample
toward runs that died early -- exactly the wrong bias for this question.
2026-09-22 06:09:16 +07:00
0xrsydn
efd7d2e423 Add batch eval and card-skip A/B scripts
eval_batch.sh: N back-to-back sessions with per-session summaries.
ab_card_skip.sh: A/B the jev vs combined card-reward skip policy and
compare deck size and progress from the game's run history.
2026-09-22 00:02:18 +07:00