sts2-bot/dataset/states/f8
0xrsydn 8fae007e50 feat(dataset): migrate game history and captures into a trainable corpus
Turn the data we already have into an open, educational dataset, so the trace we
are about to start collecting has somewhere to go.

`migrate.py` produces:
  * runs.jsonl          37 runs with outcome, killer, seed and final deck
  * decisions.jsonl     1052 decisions, each with its own outcome attached
  * states_index.jsonl  346 unique observations
  * states/             content-addressed gzipped blobs

Content addressing matters: measured, only 56% of captures are unique, so 44% of
storage is duplicates. 3.28 MB raw -> 0.42 MB stored.

The card-reward rows keep the REJECTED options, so this is a ranking dataset
rather than a classification one, and the per-fight `damage_taken` /
`turns_taken` pair is the dense reward signal a combat policy is judged on.

What it deliberately does NOT do: reconstruct per-step combat state/action pairs.
The session logs record the action but not the observation, and captures exist
only for combat, so a step has a state with no action or an action with no state
-- never both. Inventing them would poison the corpus. The gap is declared in
manifest.json instead, and collect.py will close it going forward.

Integrity checking is a separate entry point (`--check-only`) because `--verify`
alone rebuilds first and so can only ever see data that is correct by
construction -- a smoke test pretending to be a check. All six invariants were
verified by deliberately breaking the dataset and confirming the checker fails.
2026-09-22 06:09:16 +07:00
..
f8a4d49fcc5f50fa828d16fa384fec311ec8407c0a15f9c26201577336a7221a.json.gz feat(dataset): migrate game history and captures into a trainable corpus 2026-09-22 06:09:16 +07:00
f88c1981a5e80186104116c82a9b9a7a9f1582ebc6ddd07ec492b4fab9379a31.json.gz feat(dataset): migrate game history and captures into a trainable corpus 2026-09-22 06:09:16 +07:00