sts2-bot/dataset/states/62
0xrsydn 8fae007e50 feat(dataset): migrate game history and captures into a trainable corpus
Turn the data we already have into an open, educational dataset, so the trace we
are about to start collecting has somewhere to go.

`migrate.py` produces:
  * runs.jsonl          37 runs with outcome, killer, seed and final deck
  * decisions.jsonl     1052 decisions, each with its own outcome attached
  * states_index.jsonl  346 unique observations
  * states/             content-addressed gzipped blobs

Content addressing matters: measured, only 56% of captures are unique, so 44% of
storage is duplicates. 3.28 MB raw -> 0.42 MB stored.

The card-reward rows keep the REJECTED options, so this is a ranking dataset
rather than a classification one, and the per-fight `damage_taken` /
`turns_taken` pair is the dense reward signal a combat policy is judged on.

What it deliberately does NOT do: reconstruct per-step combat state/action pairs.
The session logs record the action but not the observation, and captures exist
only for combat, so a step has a state with no action or an action with no state
-- never both. Inventing them would poison the corpus. The gap is declared in
manifest.json instead, and collect.py will close it going forward.

Integrity checking is a separate entry point (`--check-only`) because `--verify`
alone rebuilds first and so can only ever see data that is correct by
construction -- a smoke test pretending to be a check. All six invariants were
verified by deliberately breaking the dataset and confirming the checker fails.
2026-09-22 06:09:16 +07:00
..
62c189f278c825f422cfa79e94391c8c8ce653ca815b6df4a1dcbc396dee2f65.json.gz feat(dataset): migrate game history and captures into a trainable corpus 2026-09-22 06:09:16 +07:00
62c3425cc516394ae5b3c6301f0f3ca39b5513149e2f7c9788a4b75813e5ae68.json.gz feat(dataset): migrate game history and captures into a trainable corpus 2026-09-22 06:09:16 +07:00