test(compat): pin independent Pong gameplay checks

This commit is contained in:
0xrsydn 2026-10-06 22:39:40 +07:00
commit 9bea613674
10 changed files with 348 additions and 0 deletions

View file

@ -7,6 +7,7 @@ The Game Boy Advance (GBA) is the first and currently only system.
It is an emulator foundation: an ARM7TDMI interpreter, memory bus, timers, DMA, Mode 0–5 video, and an original BIOS replacement. It is an emulator foundation: an ARM7TDMI interpreter, memory bus, timers, DMA, Mode 0–5 video, and an original BIOS replacement.
The included demos run original ARM test programs through the emulated hardware. The included demos run original ARM test programs through the emulated hardware.
Raw ROM files can run in terminal sessions or a window with keyboard input. Audio and saves are not implemented. Raw ROM files can run in terminal sessions or a window with keyboard input. Audio and saves are not implemented.
A [pinned Pong homebrew](docs/public-pong.md) passes scripted menu, ball-motion, and paddle-input checks.
## Layout ## Layout
@ -46,6 +47,7 @@ Use `--test-suite suite.json` for [repeatable headless ROM assertions](docs/rom-
| [docs/public-thumb-tests.md](docs/public-thumb-tests.md) | Pinned public Thumb test, result protocol, and compatibility baseline | | [docs/public-thumb-tests.md](docs/public-thumb-tests.md) | Pinned public Thumb test, result protocol, and compatibility baseline |
| [docs/public-memory-tests.md](docs/public-memory-tests.md) | Pinned public memory test, results, and video-byte coverage limits | | [docs/public-memory-tests.md](docs/public-memory-tests.md) | Pinned public memory test, results, and video-byte coverage limits |
| [docs/public-bios-tests.md](docs/public-bios-tests.md) | Pinned BIOS read-protection result and firmware limits | | [docs/public-bios-tests.md](docs/public-bios-tests.md) | Pinned BIOS read-protection result and firmware limits |
| [docs/public-pong.md](docs/public-pong.md) | Playable homebrew setup, scripted gameplay baseline, and limits |
| [docs/architecture.md](docs/architecture.md) | Crate boundaries and a file-by-file map | | [docs/architecture.md](docs/architecture.md) | Crate boundaries and a file-by-file map |
| [docs/hardware/cpu.md](docs/hardware/cpu.md) | ARM/Thumb execution rules, modes, exceptions, and timing | | [docs/hardware/cpu.md](docs/hardware/cpu.md) | ARM/Thumb execution rules, modes, exceptions, and timing |
| [docs/hardware/cartridge.md](docs/hardware/cartridge.md) | ROM loading, window/input controls, original ROM generator, and diagnostics | | [docs/hardware/cartridge.md](docs/hardware/cartridge.md) | ROM loading, window/input controls, original ROM generator, and diagnostics |

View file

@ -16,6 +16,8 @@ The repository is a Cargo workspace with three crates:
| `flake.nix`, `flake.lock` | Pinned Rust/Python development environment | | `flake.nix`, `flake.lock` | Pinned Rust/Python development environment |
| `tools/prepare_gba_tests.py`, `tools/gba-tests-{arm,thumb,memory,bios}.lock.json` | Hash-verified public test source/ROM preparation and suite generation | | `tools/prepare_gba_tests.py`, `tools/gba-tests-{arm,thumb,memory,bios}.lock.json` | Hash-verified public test source/ROM preparation and suite generation |
| `tools/test_prepare_gba_tests.py` | Offline adapter tests using original synthetic data | | `tools/test_prepare_gba_tests.py` | Offline adapter tests using original synthetic data |
| `tools/prepare_homebrew_pong.py`, `tools/homebrew-pong.lock.json` | Hash-pinned homebrew preparation and source-derived gameplay scenarios |
| `tools/test_prepare_homebrew_pong.py` | Offline homebrew preparation, provenance, bounds, and destination-safety tests |
| `.envrc` | Automatic environment loading through direnv | | `.envrc` | Automatic environment loading through direnv |
| `AGENTS.md` | Short orientation for coding agents: crate map, commands, and where docs live | | `AGENTS.md` | Short orientation for coding agents: crate map, commands, and where docs live |
| `docs/` | Project documentation; `docs/hardware/` holds per-subsystem behavior notes | | `docs/` | Project documentation; `docs/hardware/` holds per-subsystem behavior notes |

82
docs/public-pong.md Normal file
View file

@ -0,0 +1,82 @@
# Pinned public Pong gameplay baseline
ZeroDayArcade's Pong homebrew now passes scripted menu and gameplay checks on Darwin arm64.
Debug and release reports match. A bounded native window run also completes without a diagnostic.
This is a narrow gameplay baseline, not a complete playthrough or general commercial-game compatibility claim.
## Prepare and run
```sh
direnv exec . python3 -B tools/prepare_homebrew_pong.py /tmp/gba-public-pong-pinned
direnv exec . cargo run --locked --release -- --test-suite /tmp/gba-public-pong-pinned/suite.json > /tmp/gba-pong-report.json
direnv exec . cargo run --locked --release -- --rom /tmp/gba-public-pong-pinned/Pong-Homebrew-GBA.gba --window
```
The preparation directory must not already exist. Choose another path for another preparation.
Use `--source-dir PATH` for an offline copy containing every file listed in the lock.
Every source, license, build file, and ROM must match its pinned size and SHA-256 hash before output is created.
Output must stay outside the repository or inside ignored `roms/`. Existing files are never overwritten.
A disk error can leave a partial output directory. Preparation alone does not run or pass the suite.
Window controls:
- Enter or Z selects a menu item (GBA Start or A).
- Up/Down changes the menu selection and moves the player paddle during a rally.
- Escape closes the window.
The game deliberately pauses before the first rally and after scores. Paddle movement starts after the initial pause.
Click the window if it does not receive keyboard input.
The game has no sound or cartridge saves. It does not validate either missing subsystem.
## Provenance
- Repository: [ZeroDayArcade/Pong-Homebrew-GBA](https://github.com/ZeroDayArcade/Pong-Homebrew-GBA).
- Revision: `c784b6036a4f188c50932b411e98126bfcbd07d6`.
- License: MIT; the adapter retains the upstream license in the local output directory.
- Lock: [`tools/homebrew-pong.lock.json`](../tools/homebrew-pong.lock.json).
- ROM: upstream `Pong-Homebrew-GBA.gba`, not a locally rebuilt binary.
The repository contains only original adapter logic, sparse assertions, metadata, and documentation.
No upstream font arrays, implementation source, ROM bytes, or other assets are included.
Build-source correspondence is not established through a reproducible rebuild; the committed upstream ROM is hash-pinned separately.
## Assertion design
All cases start from a fresh machine with the original BIOS replacement.
The manifest uses [version 2 gameplay checks](rom-tests.md#manifest-version-2-gameplay-checks), with a separate machine-step budget for every case.
Button snapshots apply after the numbered VBlank capture and remain held until replaced.
Coordinates and colors come from the pinned `main.c`, `graphics.h`, and font definitions, not captured-image golden files.
| Case | Completion VBlank | Evidence |
| --- | --- | --- |
| Menu | 30 | Title pixel and Play cursor visible; Settings cursor absent |
| Settings | 50 | Down/A changes the heading to the rules screen and moves the cursor |
| Back | 70 | Start returns to the main heading with Settings still selected |
| Start | 40 | Start displays both blue paddles, the green ball, and the white center line |
| Ball motion | 160 | Initial ball point is cleared and a later point is green |
| Up | 175 | Holding Up moves the paddle above its initial position |
| Release | 185 | Releasing Up retains that position instead of continuing movement |
| Down | 205 | Holding Down returns the paddle to its initial vertical range |
Game-start cases press Start at VBlank 30 and release at 32.
The source initializes a 120-iteration pause before ball movement.
The motion checks sample interior pixels to tolerate a small draw-boundary difference; they do not establish exact CPU timing.
Up is held from VBlank 160 to 170. Down is held from 190 to 200.
At two pixels per update, these intervals move the paddle by 20 pixels in opposite directions.
The release case checks both occupied and cleared points to detect continued movement.
## Failure found and corrected
Before the [BIOS boot graphics correction](research/boot-video.md), all gameplay cases failed their visible-pixel assertions.
CPU-only runs had reached the VBlank loop without an error, which concealed the black-screen failure.
The ROM selected Mode 3 without setting affine coefficients. Our firmware had left its scale at zero.
Original ARM boot stores now initialize BG2/BG3 identity scales. No ROM patch or game-specific renderer behavior was added.
The same pinned ROM and assertions pass after the correction.
## Limits and next coverage
Sparse pixels do not validate every glyph, complete frames, collision behavior, scoring, or the victory/reset loop.
The native-window run checks startup and presentation, not a manual keyboard playthrough.
Automated input uses the core button interface. Original native-window tests separately check keyboard mapping and session behavior.
The baseline exercises Mode 3, libgba startup, Thumb/ARM execution, VBlank IRQs, BIOS waits, and button edges.
Select another independent program to expose different graphics, audio, and cartridge requirements.

View file

@ -191,5 +191,6 @@ For further public ARM7TDMI/GBA tests, inspect their source, license, entry assu
Pin source revisions and build instructions before comparing emulator changes. Pin source revisions and build instructions before comparing emulator changes.
Adapt suites to verified completion addresses and result locations; do not guess them or treat timeouts as passes. Adapt suites to verified completion addresses and result locations; do not guess them or treat timeouts as passes.
Version 2 supports scheduled input and exact captured-pixel checks. Version 2 supports scheduled input and exact captured-pixel checks.
The [pinned public Pong scenarios](public-pong.md) now exercise menus, ball movement, and paddle press/release behavior.
Debug-port logging, external firmware selection, audio assertions, and whole-image comparisons remain unsupported. Debug-port logging, external firmware selection, audio assertions, and whole-image comparisons remain unsupported.
Keep external binaries local. Record consulted sources in [references.md](references.md). Keep external binaries local. Record consulted sources in [references.md](references.md).

View file

@ -82,6 +82,8 @@ Update this file when a feature lands or a limit is removed.
- Headless JSON ROM suites with PC or VBlank completion, register/CPSR/memory/pixel assertions, scheduled buttons, bounded budgets, and structured failure reports. - Headless JSON ROM suites with PC or VBlank completion, register/CPSR/memory/pixel assertions, scheduled buttons, bounded budgets, and structured failure reports.
- Hash-pinned preparation of public `jsmolka/gba-tests` ARM, Thumb, memory, and BIOS ROMs, with verified result-register checkpoints. - Hash-pinned preparation of public `jsmolka/gba-tests` ARM, Thumb, memory, and BIOS ROMs, with verified result-register checkpoints.
- These pinned ROMs pass their checkpoints on Darwin arm64 in debug and release builds; coverage limits remain documented. - These pinned ROMs pass their checkpoints on Darwin arm64 in debug and release builds; coverage limits remain documented.
- A [pinned MIT Pong homebrew](public-pong.md) passes captured-pixel menu, ball-motion, and paddle press/release checks in both profiles.
A bounded native-window run also completes; scoring, full playthrough, audio, and saves are not validated by this ROM.
- The original BIOS exposes documented boot, SoftReset, SWI, and IRQ protected-read words through its actual exit layout. - The original BIOS exposes documented boot, SoftReset, SWI, and IRQ protected-read words through its actual exit layout.
- ARM test/compare Rd=15 status restoration, without a PC result write or nominal refill. - ARM test/compare Rd=15 status restoration, without a PC result write or nominal refill.
- ARM unused-memory data reads return a PC+8 word snapshot, with normal byte lanes and load rotation. - ARM unused-memory data reads return a PC+8 word snapshot, with normal byte lanes and load rotation.

View file

@ -62,6 +62,8 @@ direnv exec . python3 -B -m unittest discover -s tools -p 'test_*.py' -v
``` ```
They verify suite selection, pinned URLs, hashes, sizes, checkpoint validation, offline copies, destination safety, and non-overwriting output. They verify suite selection, pinned URLs, hashes, sizes, checkpoint validation, offline copies, destination safety, and non-overwriting output.
The [Pong adapter](public-pong.md) has additional offline tests for its pinned downloads, retained license, and bounded gameplay scenarios.
Run the external Pong suite separately; ordinary Cargo and Python tests require no public ROM or network access.
Thumb cases also verify the return-to-ARM bridge. Command-line tests cover default ARM selection, explicit Thumb/memory/BIOS selection, and errors. Thumb cases also verify the return-to-ARM bridge. Command-line tests cover default ARM selection, explicit Thumb/memory/BIOS selection, and errors.
Compare/status CPU regressions verify saved flags/masks, bank switching, sequential PC/timing, User/System fallback, and atomic diagnostics. Compare/status CPU regressions verify saved flags/masks, bank switching, sequential PC/timing, User/System fallback, and atomic diagnostics.
Load-alias regressions cover widths, signed/unaligned values, immediate/register offsets, pre/post-indexing, every non-PC register bank, and address wrapping. Load-alias regressions cover widths, signed/unaligned values, immediate/register offsets, pre/post-indexing, every non-PC register bank, and address wrapping.

View file

@ -33,6 +33,14 @@ SRAM, Flash, and EEPROM require different protocols. Supporting one does not est
Audio needs device clocks, mixing, and buffering; accepting sound-register writes alone is not audio support. Audio needs device clocks, mixing, and buffering; accepting sound-register writes alone is not audio support.
The detailed hardware limits remain in [status.md](status.md). The detailed hardware limits remain in [status.md](status.md).
## Current gameplay baseline
The [pinned Pong homebrew](public-pong.md) passes scripted menus, ball movement, and paddle press/release checks.
The native Darwin window also starts and presents frames without a diagnostic.
This exposed and corrected missing firmware affine-scale initialization, without a game-specific patch.
Frame-based scenarios and the preparation adapter now make the result repeatable.
Next, select an independent game with different device requirements. Cartridge saves and audio remain unimplemented.
## Development loop ## Development loop
For each feature, inspect the source and hardware evidence, then write original regression tests. For each feature, inspect the source and hardware evidence, then write original regression tests.

View file

@ -0,0 +1,36 @@
{
"repository": "ZeroDayArcade/Pong-Homebrew-GBA",
"revision": "c784b6036a4f188c50932b411e98126bfcbd07d6",
"license": "MIT",
"rom": "Pong-Homebrew-GBA.gba",
"files": {
"LICENSE": {
"bytes": 1074,
"sha256": "8e466e1d75cd5e8c1deb5397de6275429d0046f62d5c59129e2e6872559b9b67"
},
"README.md": {
"bytes": 6838,
"sha256": "6ea1beefe845bacf9fa443a1dad3c952ad43fa02596c080617f68088c11ab66c"
},
"Makefile": {
"bytes": 6229,
"sha256": "f1185b69995f8a9e6be21457eea0ea5ca8dcfc5f132e77a2e49069e81c572d33"
},
"source/main.c": {
"bytes": 12987,
"sha256": "8f757309fa142b3d3a4b369a48c20a967958182c22a643456d790549a46bfce4"
},
"source/graphics.h": {
"bytes": 5305,
"sha256": "0ee9e111ea484e8e4062693e5e35c32117981968ee32eafc5f3bab1d3925bfea"
},
"source/characters.h": {
"bytes": 11247,
"sha256": "216a7457764b180eeeba239526646e0e658c8a5283b7025cc8e4da4ed62a9c1d"
},
"Pong-Homebrew-GBA.gba": {
"bytes": 8884,
"sha256": "3c850319ddef6d237756dd64b85ddf3b75bf7c096feb2408ed1b2dd0a05a0158"
}
}
}

View file

@ -0,0 +1,99 @@
#!/usr/bin/env python3
"""Prepare hash-pinned public Pong gameplay checks outside tracked paths.
Only original adapter logic is included here. Downloaded source, assets, ROM,
and license stay in the requested local directory. A successful preparation
is not a compatibility verdict. Run the generated suite separately.
"""
import argparse
import json
from pathlib import Path
import sys
from prepare_gba_tests import MAX_FILE_BYTES, destination, fetch, verify
LOCK = Path(__file__).with_name("homebrew-pong.lock.json")
def manifest(rom):
"""Sparse assertions derived from the pinned source, not emulator screenshots."""
white, blue, green = 0xffffff, 0x0000ff, 0x00ff00
def pixel(x, y, color):
return {"kind": "pixel", "x": x, "y": y, "equals": color}
def inputs(*pairs):
return [{"vblank": frame, "buttons": buttons} for frame, buttons in pairs]
def case(name, end, events, checks):
return {"name": f"pong-{name}", "rom": rom, "step_limit": 3_000_000,
"completion": {"vblanks": end}, "inputs": events, "checks": checks}
start = inputs((30, 8), (32, 0)) # Start, release
move = start + inputs((160, 64), (170, 0)) # Up, release after initial pause
menu = inputs((30, 128), (32, 0), (40, 1), (42, 0)) # Down, A
return {"version": 2, "cases": [
case("menu", 30, [], [pixel(81, 64, white), pixel(74, 88, white), pixel(74, 100, 0)]),
case("settings", 50, menu, [pixel(81, 64, 0), pixel(84, 64, white),
pixel(74, 88, 0), pixel(74, 100, white)]),
case("back", 70, menu + inputs((60, 8), (62, 0)),
[pixel(81, 64, white), pixel(74, 100, white)]),
case("start", 40, start, [pixel(2, 70, blue), pixel(232, 70, blue),
pixel(118, 77, green), pixel(120, 2, white)]),
case("ball-motion", 160, start, [pixel(118, 77, 0), pixel(135, 95, green)]),
case("up", 175, move, [pixel(2, 50, blue), pixel(2, 90, 0)]),
case("release", 185, move, [pixel(2, 50, blue), pixel(2, 90, 0)]),
case("down", 205, move + inputs((190, 128), (200, 0)),
[pixel(2, 50, 0), pixel(2, 70, blue)]),
]}
def prepare(output, source=None, download=fetch):
output = destination(output)
lock = json.loads(LOCK.read_text(encoding="utf-8"))
base = f"https://raw.githubusercontent.com/{lock['repository']}/{lock['revision']}/"
files = {}
for name, expected in lock["files"].items():
if source is None:
data = download(base + name)
else:
path = Path(source) / name
if not path.is_file() or path.stat().st_size > MAX_FILE_BYTES:
raise ValueError(f"source must be a bounded regular file: {name}")
with path.open("rb") as stream:
data = stream.read(MAX_FILE_BYTES + 1)
verify(name, data, expected)
files[name] = data
# Verify the full pinned set before creating output. Never reuse an existing
# directory or overwrite files. Disk errors can leave partial output.
output.mkdir()
for name, data in files.items():
path = output / name
path.parent.mkdir(parents=True, exist_ok=True)
with path.open("xb") as stream:
stream.write(data)
for name, value in [("suite.json", manifest(lock["rom"])), ("source-lock.json", lock)]:
with (output / name).open("x", encoding="utf-8") as stream:
json.dump(value, stream, indent=2)
stream.write("\n")
return output / "suite.json"
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("output", type=Path, help="new directory outside the repository or under roms/")
parser.add_argument("--source-dir", type=Path, help="offline upstream files matching every pinned hash")
args = parser.parse_args()
try:
path = prepare(args.output, args.source_dir)
except (OSError, ValueError) as error:
print(f"cannot prepare homebrew Pong: {error}", file=sys.stderr)
return 1
print(f"Prepared pinned Pong suite: {path}")
print("Run with gameboy-rust --test-suite PATH. Preparation does not mean the tests passed.")
return 0
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,114 @@
"""Offline adapter tests. All fixture bytes are original synthetic data."""
import hashlib
import json
from pathlib import Path
import tempfile
import unittest
from unittest.mock import patch
import prepare_homebrew_pong as adapter
from prepare_gba_tests import ROOT, MAX_FILE_BYTES
class PongPreparation(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.addCleanup(self.temp.cleanup)
self.base = Path(self.temp.name)
self.output = self.base / "prepared"
self.source = self.base / "source"
self.source.mkdir()
real = json.loads(adapter.LOCK.read_text())
self.files = {name: f"original fixture: {name}\n".encode() for name in real["files"]}
self.lock = dict(real, files={name: {"bytes": len(data), "sha256": hashlib.sha256(data).hexdigest()}
for name, data in self.files.items()})
self.lock_path = self.base / "lock.json"
self.lock_path.write_text(json.dumps(self.lock))
self.addCleanup(patch.stopall)
patch.object(adapter, "LOCK", self.lock_path).start()
for name, data in self.files.items():
path = self.source / name
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(data)
def test_offline_preparation_retains_license_and_is_repeatable(self):
first = adapter.prepare(self.output, self.source)
second = adapter.prepare(self.base / "second", self.source)
self.assertEqual(first.read_bytes(), second.read_bytes())
self.assertEqual(json.loads(first.read_text()), adapter.manifest(self.lock["rom"]))
self.assertEqual(json.loads((self.output / "source-lock.json").read_text()), self.lock)
for name, data in self.files.items():
self.assertEqual((self.output / name).read_bytes(), data)
with self.assertRaisesRegex(ValueError, "already exists"):
adapter.prepare(self.output, self.source)
self.assertEqual((self.output / "LICENSE").read_bytes(), self.files["LICENSE"])
def test_download_uses_only_pinned_revision_urls(self):
base = f"https://raw.githubusercontent.com/{self.lock['repository']}/{self.lock['revision']}/"
calls = []
def download(url):
self.assertTrue(url.startswith(base))
calls.append(url.removeprefix(base))
return self.files[calls[-1]]
adapter.prepare(self.output, download=download)
self.assertEqual(calls, list(self.files))
def test_bad_hash_size_missing_and_oversized_files_leave_no_output(self):
rom = self.source / self.lock["rom"]
original = rom.read_bytes()
for replacement in [bytes(len(original)), original + b"x", b"x" * (MAX_FILE_BYTES + 1)]:
rom.write_bytes(replacement)
with self.assertRaises(ValueError):
adapter.prepare(self.output, self.source)
self.assertFalse(self.output.exists())
rom.unlink()
with self.assertRaises(ValueError):
adapter.prepare(self.output, self.source)
self.assertFalse(self.output.exists())
def test_network_failure_leaves_no_output(self):
def fail(_):
raise OSError("offline")
with self.assertRaises(OSError):
adapter.prepare(self.output, download=fail)
self.assertFalse(self.output.exists())
def test_tracked_and_symlinked_destinations_are_rejected_before_fetch(self):
def fail(_):
self.fail("must not fetch for unsafe destination")
with self.assertRaises(ValueError):
adapter.prepare(ROOT / "tracked-pong-test", download=fail)
link = self.base / "linked"
link.symlink_to(ROOT, target_is_directory=True)
with self.assertRaises(ValueError):
adapter.prepare(link / "tracked-pong-test", download=fail)
with self.assertRaises(ValueError):
adapter.prepare(self.base / "missing" / "out", download=fail)
def test_manifest_has_bounded_distinct_scenarios_and_complete_button_snapshots(self):
manifest = adapter.manifest("test.gba")
self.assertEqual(manifest["version"], 2)
cases = manifest["cases"]
self.assertEqual(len({case["name"] for case in cases}), len(cases))
self.assertLessEqual(sum(case["step_limit"] for case in cases), 100000000)
for case in cases:
self.assertEqual(case["rom"], "test.gba")
frames = [event["vblank"] for event in case["inputs"]]
self.assertEqual(frames, sorted(set(frames)))
self.assertTrue(all(frame < case["completion"]["vblanks"] for frame in frames))
for event in case["inputs"]:
self.assertLessEqual(event["buttons"], 1023)
if case["inputs"]:
self.assertEqual(case["inputs"][-1]["buttons"], 0)
self.assertTrue(case["checks"])
for check in case["checks"]:
self.assertEqual(check["kind"], "pixel")
self.assertLess(check["x"], 240)
self.assertLess(check["y"], 160)
self.assertLessEqual(check["equals"], 0xffffff)
if __name__ == "__main__":
unittest.main()