sts2-bot/dataset/states/7f
0xrsydn 8fae007e50 feat(dataset): migrate game history and captures into a trainable corpus
Turn the data we already have into an open, educational dataset, so the trace we
are about to start collecting has somewhere to go.

`migrate.py` produces:
  * runs.jsonl          37 runs with outcome, killer, seed and final deck
  * decisions.jsonl     1052 decisions, each with its own outcome attached
  * states_index.jsonl  346 unique observations
  * states/             content-addressed gzipped blobs

Content addressing matters: measured, only 56% of captures are unique, so 44% of
storage is duplicates. 3.28 MB raw -> 0.42 MB stored.

The card-reward rows keep the REJECTED options, so this is a ranking dataset
rather than a classification one, and the per-fight `damage_taken` /
`turns_taken` pair is the dense reward signal a combat policy is judged on.

What it deliberately does NOT do: reconstruct per-step combat state/action pairs.
The session logs record the action but not the observation, and captures exist
only for combat, so a step has a state with no action or an action with no state
-- never both. Inventing them would poison the corpus. The gap is declared in
manifest.json instead, and collect.py will close it going forward.

Integrity checking is a separate entry point (`--check-only`) because `--verify`
alone rebuilds first and so can only ever see data that is correct by
construction -- a smoke test pretending to be a check. All six invariants were
verified by deliberately breaking the dataset and confirming the checker fails.
2026-09-22 06:09:16 +07:00
..
7f04b01bdb43927894b7d76136efae5d8655893ddcda3e7633cbd2b8042b4d5b.json.gz feat(dataset): migrate game history and captures into a trainable corpus 2026-09-22 06:09:16 +07:00
7fc40d0d5e0200e7a7c98bb2ca1be5654db43384bddba091914891945cc29a27.json.gz feat(dataset): migrate game history and captures into a trainable corpus 2026-09-22 06:09:16 +07:00
7fe8a41d1ea2f2ad9048f5000b77d4288394725b061b9954940977c848c5c85b.json.gz feat(dataset): migrate game history and captures into a trainable corpus 2026-09-22 06:09:16 +07:00