sts2-bot/dataset/states/7b
0xrsydn 8fae007e50 feat(dataset): migrate game history and captures into a trainable corpus
Turn the data we already have into an open, educational dataset, so the trace we
are about to start collecting has somewhere to go.

`migrate.py` produces:
  * runs.jsonl          37 runs with outcome, killer, seed and final deck
  * decisions.jsonl     1052 decisions, each with its own outcome attached
  * states_index.jsonl  346 unique observations
  * states/             content-addressed gzipped blobs

Content addressing matters: measured, only 56% of captures are unique, so 44% of
storage is duplicates. 3.28 MB raw -> 0.42 MB stored.

The card-reward rows keep the REJECTED options, so this is a ranking dataset
rather than a classification one, and the per-fight `damage_taken` /
`turns_taken` pair is the dense reward signal a combat policy is judged on.

What it deliberately does NOT do: reconstruct per-step combat state/action pairs.
The session logs record the action but not the observation, and captures exist
only for combat, so a step has a state with no action or an action with no state
-- never both. Inventing them would poison the corpus. The gap is declared in
manifest.json instead, and collect.py will close it going forward.

Integrity checking is a separate entry point (`--check-only`) because `--verify`
alone rebuilds first and so can only ever see data that is correct by
construction -- a smoke test pretending to be a check. All six invariants were
verified by deliberately breaking the dataset and confirming the checker fails.
2026-09-22 06:09:16 +07:00
..
7b18fbd98d72724b030a555bd0ccf48e5be44ca02a1cea4047ddb0d647114f83.json.gz feat(dataset): migrate game history and captures into a trainable corpus 2026-09-22 06:09:16 +07:00
7bd6396ce7ed49d4d11cd3f4310aa1bcf3efc0f791c4ef8aabae4759f9f6390d.json.gz feat(dataset): migrate game history and captures into a trainable corpus 2026-09-22 06:09:16 +07:00
7bf9d4ce6b4167910d7f866e381339367651ba0219527d528dfadead30dcb920.json.gz feat(dataset): migrate game history and captures into a trainable corpus 2026-09-22 06:09:16 +07:00