diff --git a/.gitignore b/.gitignore index 765251a..5348850 100644 --- a/.gitignore +++ b/.gitignore @@ -4,3 +4,5 @@ *.gba *.nds *.sav +__pycache__/ +*.pyc diff --git a/AGENTS.md b/AGENTS.md index 098e731..33111de 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -34,6 +34,7 @@ Window smoke tests (`--*-smoke-test`) need a logged-in desktop session. Details | Working on | Read | | --- | --- | | ARM/Thumb instructions, modes, exceptions, timing | [docs/hardware/cpu.md](docs/hardware/cpu.md) | +| Pinned public ARM tests and known compatibility failures | [docs/public-arm-tests.md](docs/public-arm-tests.md) | | ROM test suites, assertions, and JSON reports | [docs/rom-tests.md](docs/rom-tests.md) | | ROM files, window execution, and terminal diagnostics | [docs/hardware/cartridge.md](docs/hardware/cartridge.md) | | BIOS services (SWI) | [docs/hardware/bios.md](docs/hardware/bios.md) | diff --git a/README.md b/README.md index 5675254..d9df73b 100644 --- a/README.md +++ b/README.md @@ -41,6 +41,7 @@ Use `--test-suite suite.json` for [repeatable headless ROM assertions](docs/rom- | [docs/development.md](docs/development.md) | Nix, direnv, and macOS build notes | | [docs/testing.md](docs/testing.md) | Validation commands and what the test suites cover | | [docs/rom-tests.md](docs/rom-tests.md) | Headless ROM suites, completion checks, assertions, and JSON reports | +| [docs/public-arm-tests.md](docs/public-arm-tests.md) | Pinned public ARM test, preparation command, and known failures | | [docs/architecture.md](docs/architecture.md) | Crate boundaries and a file-by-file map | | [docs/hardware/cpu.md](docs/hardware/cpu.md) | ARM/Thumb execution rules, modes, exceptions, and timing | | [docs/hardware/cartridge.md](docs/hardware/cartridge.md) | ROM loading, window/input controls, original ROM generator, and diagnostics | diff --git a/docs/architecture.md b/docs/architecture.md index 282f007..63d7b32 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -13,7 +13,9 @@ The repository is a Cargo workspace with three crates: | Path | Purpose | | --- | --- | -| `flake.nix`, `flake.lock` | Pinned development environment | +| `flake.nix`, `flake.lock` | Pinned Rust/Python development environment | +| `tools/prepare_gba_tests.py`, `tools/gba-tests-arm.lock.json` | Hash-verified public ARM source/ROM preparation and suite generation | +| `tools/test_prepare_gba_tests.py` | Offline adapter tests using original synthetic data | | `.envrc` | Automatic environment loading through direnv | | `AGENTS.md` | Short orientation for coding agents: crate map, commands, and where docs live | | `docs/` | Project documentation; `docs/hardware/` holds per-subsystem behavior notes | @@ -24,6 +26,7 @@ The repository is a Cargo workspace with three crates: | `crates/gba-core/src/cpu/thumb.rs` | Thumb decoding and execution | | `crates/gba-core/src/cpu/status.rs` | Processor modes, register banks, and status transfers | | `crates/gba-core/src/cpu/exception.rs` | Reset state, exception entry, and interrupt sampling | +| `crates/gba-core/src/cpu/compare_psr_tests.rs` | Original Rd=15 test/compare status restoration, flags, bank, and timing regressions | | `crates/gba-core/src/cpu/status_tests.rs`, `crates/gba-core/src/cpu/exception_tests.rs` | Status, banking, exception, and return tests | | `crates/gba-core/src/cpu/thumb_tests.rs` | Thumb instructions, state switching, and decoder checks | | `crates/gba-core/src/cpu/transfer_tests.rs` | Stack, addressing, overlap, empty-list, and swap tests | diff --git a/docs/development.md b/docs/development.md index ef7ea4f..1e4bbbf 100644 --- a/docs/development.md +++ b/docs/development.md @@ -54,7 +54,8 @@ direnv exec . cargo run --locked direnv exec . cargo test --locked ``` -`.envrc` uses the Nix flake to load Rust, Cargo, rustfmt, and Clippy. +`.envrc` uses the Nix flake to load Rust, Cargo, rustfmt, Clippy, and Python 3. +Python supports the pinned public-test preparation adapter and its standard-library-only tests. `flake.lock` pins the Nix package source. Keep it in version control. The first run needs network access to download the development tools and Rust dependencies. diff --git a/docs/public-arm-tests.md b/docs/public-arm-tests.md new file mode 100644 index 0000000..6a8ffc7 --- /dev/null +++ b/docs/public-arm-tests.md @@ -0,0 +1,98 @@ +# Pinned public ARM test + +The headless runner now executes the unmodified ARM ROM from [jsmolka/gba-tests](https://github.com/jsmolka/gba-tests). +This is independent public test code, not a complete ARM7TDMI conformance suite or a new hardware measurement. +The full public ARM test still fails. The first discovered CPU error is fixed; the next unsupported instruction is recorded below. + +## Prepare and run + +```sh +direnv exec . python3 tools/prepare_gba_tests.py /tmp/gba-public-arm +direnv exec . cargo run --locked --release -- --test-suite /tmp/gba-public-arm/suite.json > /tmp/gba-public-arm-report.json +``` + +The destination must not exist. Its parent directory must exist. +Use a directory outside the repository or within the ignored `roms/` directory. +The adapter downloads pinned files over HTTPS and checks each byte count and SHA-256 digest before creating output. +It retains the upstream MIT license, README, ARM sources, shared assembly includes, and the original ROM. +It also writes `suite.json` and `source-lock.json`. The manifest references the ROM relative to its own directory. +No source rewriting, header removal, binary patching, or expected-result adjustment occurs. + +Python comes from the existing pinned Nix environment. No Python packages are required. +The adapter bounds each download, sets a network timeout, and refuses existing output directories. +A disk-write failure can leave partial output; choose a new destination for another attempt. +Ordinary Rust and Python unit tests do not download or execute external ROMs. + +For an offline copy of the selected upstream files: + +```sh +direnv exec . python3 tools/prepare_gba_tests.py /tmp/gba-public-arm-offline --source-dir /path/to/gba-tests +``` + +Offline input must match every selected file in the lock. A directory from a different revision can fail verification. +The adapter does not overwrite or modify the input tree. + +## Provenance and checkpoint + +- Repository: `jsmolka/gba-tests`. +- Revision: `a7113b67e63f83a9b321696ddd7042ccfad6c881`. +- Upstream project license: MIT, copyright Julian Smolka. The license is retained with downloaded files. +- ROM: `arm/arm.gba`, 8,824 bytes. +- SHA-256: `77ee88662552bdc885c1080c0172ff119d54db791bd73b21808cf1ff1fe5b40e`. +- All selected file hashes and sizes: [`tools/gba-tests-arm.lock.json`](../tools/gba-tests-arm.lock.json). + +This adapter uses the committed upstream binary, not a local rebuild. +Upstream documents FASMARM as its assembler. No FASMARM build or source-to-binary reproduction was performed on Darwin. +The binary's console header includes logo bytes. Do not copy the downloaded binary or header source into tracked files. +Only original adapter code and provenance metadata are committed here. + +The ARM source initializes r12 to zero before testing. +Each failure path sets r12 to its test number and branches to `eval`. +Successful execution falls through to the same label with r12 still zero. +At this pinned revision, `eval` is `0x08001d4c`, in ARM state. +The ROM word there is the initial register save of the `m_vsync` macro. +The source failure exits and their encoded branch targets were inspected against that address. + +The manifest stops before executing `eval` and requires `r12 == 0`. +This avoids depending on the result renderer, VBlank polling, or BIOS division used to print failure numbers. +It does not bypass the ROM's startup code or any CPU tests. +In particular, evaluation must stop before using a possibly invalid stack left by a failed mode-switch test. +The instruction at the checkpoint is checked during preparation, in addition to whole-file hash verification. + +The case budget is 1,000,000 successful machine steps, including our original BIOS boot. +Timeouts, emulator diagnostics, and nonzero r12 all remain failures under the [normal suite rules](rom-tests.md). +Preparation does not mean the test passed. The current run exits nonzero and writes a failed JSON report. + +## Observed results on Darwin arm64 + +Before the compare/status fix: + +- The ROM reaches `eval` after 579 successful steps. +- The assertion reports r12 = 234, in FIQ mode. +- Running beyond `eval` produces an unmapped stack access because the CPU incorrectly remains in FIQ mode. +- Upstream test 234 uses an ARM compare encoding with unused destination bits set to R15. + +The CPU previously treated that encoding as an ordinary compare and only changed arithmetic flags. +It now restores CPSR from the current mode's SPSR when one exists, without writing a result to PC. +User/System forms still use ordinary test/compare flags because those modes have no SPSR. +Original regressions cover all four test/compare operations, exception banks, flags, masks, operands, timing, conditions, and invalid saved modes. +See [CPU status behavior](hardware/cpu.md#processor-status-and-exceptions). + +After the fix: + +- Execution progresses past tests 234 and 235 and the PSR-transfer section. +- After 861 successful steps, the runner reports unsupported instruction `0xe5b00004` at `0x0800132c`. +- This is `LDR r0, [r0, #4]!` in upstream test 360, with base and destination equal. +- The CPU is in System/ARM state, and r12 remains zero. The final completion checkpoint has not been reached. + +The load/writeback alias case is the next compatibility task. Do not weaken the assertion or label this run a pass. +No public Thumb, timing, memory, BIOS, graphics, or unsafe suite has been claimed as passing. +These results do not establish Pokémon Emerald compatibility. + +## References + +- [Pinned ARM entry and result flow](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/arm/arm.asm). +- [Pinned test/result macros](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/lib/macros.inc). +- [Tests 234 and 235](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/arm/data_processing.asm). +- [Load/writeback alias tests](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/arm/single_transfer.asm). +- [Pinned license](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/LICENSE). diff --git a/docs/rom-tests.md b/docs/rom-tests.md index 6effa1c..0dba759 100644 --- a/docs/rom-tests.md +++ b/docs/rom-tests.md @@ -4,7 +4,8 @@ Use `--test-suite PATH.json` to run repeatable ROM checks without opening a wind Each case has a machine-step budget, a completion address, and explicit assertions. A timeout never counts as a pass. Existing `--rom ... --steps ...` remains a diagnostic mode, not a test verdict. -This runner currently has regression coverage from original programs, not independent public hardware-test results. +Original programs cover the runner itself. A [pinned public ARM test](public-arm-tests.md) now provides an independent compatibility baseline. +That public ROM still fails; passing the runner's unit tests does not mean the external ARM suite passes. A passing suite establishes only its specified checkpoints and assertions. It does not establish Emerald compatibility. ## Run the original smoke suite @@ -129,7 +130,9 @@ They do not hash ROM contents; record source revisions and file checksums alongs ## Next validation stage -Select public ARM7TDMI/GBA tests and inspect their source, license, entry assumptions, and result protocol. +The first public integration is [jsmolka/gba-tests ARM](public-arm-tests.md), with a verified result-register checkpoint and a known failing baseline. +Fix its next unsupported load/writeback alias case before claiming the full ARM ROM passes. +For further public ARM7TDMI/GBA tests, inspect their source, license, entry assumptions, and result protocol. Pin source revisions and build instructions before comparing emulator changes. Adapt suites to verified completion addresses and result locations; do not guess them or treat timeouts as passes. Some tests need debug-port logging, input, different firmware behavior, or rendered-image checks that this runner does not support yet. diff --git a/docs/status.md b/docs/status.md index 2601999..2795a3e 100644 --- a/docs/status.md +++ b/docs/status.md @@ -66,6 +66,8 @@ Update this file when a feature lands or a limit is removed. - ROM windows with scanline capture, keyboard input, focus-loss release, bounded execution slices, and interactive keypad wake from STOP. - Optional ROM-window frame limits and an original input-test ROM generator. - Headless JSON ROM suites with per-case checkpoints, register/CPSR/memory assertions, bounded budgets, and structured failure reports. +- Hash-pinned preparation of the public `jsmolka/gba-tests` ARM ROM, with an explicit failing compatibility baseline. +- ARM test/compare Rd=15 status restoration, without a PC result write or nominal refill. - Little-endian byte, halfword, and word reads/writes. - ARM7TDMI word-load rotation, aligned stores, and odd-address halfword-load behavior. - Explicit errors for unsupported instructions when their condition passes, and for unsupported memory accesses. @@ -110,7 +112,8 @@ ROM files can run in bounded terminal mode or a window with the original BIOS re ROM windows have no demo-specific startup checks. Input is sampled between bounded execution slices. Audio, saves, and external BIOS loading remain unavailable. ROM loading does not establish commercial-game compatibility. See [cartridge loading](hardware/cartridge.md) for file limits, step budgets, and exit status. -The [ROM suite runner](rom-tests.md) currently uses original regression programs, not independent public hardware-test results. +The [ROM suite runner](rom-tests.md) has original regressions and a [pinned public ARM baseline](public-arm-tests.md). +The public ARM ROM now gets past its compare/status tests but stops at test 360, a load/writeback base-destination alias. It requires known completion addresses and has no debug-port protocol, scripted input, or rendered-image assertions. Unmapped reads return errors instead of hardware open-bus values. @@ -121,7 +124,8 @@ Writes to cartridge addresses return errors instead of modeling cartridge hardwa ## Next steps -1. Adapt the ROM suite runner to pinned public ARM7TDMI test programs and verified result protocols before claiming instruction compatibility. +1. Implement verified load/writeback alias behavior exposed by public ARM test 360; rerun the pinned suite and fix subsequent failures. + Add further public suites only with verified result protocols before claiming instruction compatibility. 2. Refine nominal timing with a fetch pipeline, Game Pak prefetch, per-access device updates, and verified timer/IRQ delays. 3. Validate keypad retrigger behavior with hardware tests and add remaining device registers; extend BIOS reset coverage as sound/serial support becomes available. Validate STOP entry/wake edges and add external wake sources and remaining DMA device modes. diff --git a/docs/testing.md b/docs/testing.md index 9599e54..43e0a77 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -49,7 +49,15 @@ Do not set a Linux cross-compilation target for this validation. The [headless ROM suite runner](rom-tests.md) provides explicit checkpoint and assertion results through `--test-suite PATH.json`. Its tests verify manifest limits, whole-suite validation, exact step boundaries, ARM/Thumb checkpoints, and register/CPSR/memory checks. Additional tests cover DMA/IRQ accounting, HALT/STOP, diagnostic state, case isolation, repeatable JSON output, and failure exit status. -These original tests validate the runner; they are not independent public hardware-test results. +These original tests validate the runner. The [public ARM adapter](public-arm-tests.md) is a separate, currently failing compatibility baseline. +Its preparation tests use only synthetic original bytes and no network: + +```sh +direnv exec . python3 -B -m unittest discover -s tools -p 'test_*.py' -v +``` + +They verify pinned URLs, hashes, sizes, checkpoint validation, offline copies, destination safety, and non-overwriting output. +Compare/status CPU regressions verify saved flags/masks, bank switching, sequential PC/timing, User/System fallback, and atomic diagnostics. ROM-runner tests use temporary files containing original instructions, without Nintendo assets or a desktop session. They check file-size boundaries, bounded reads, short reads, I/O errors, paths, symlinks, and read-only loading. diff --git a/flake.nix b/flake.nix index b61b23c..8c7e432 100644 --- a/flake.nix +++ b/flake.nix @@ -11,7 +11,7 @@ let pkgs = import nixpkgs { inherit system; }; in { default = pkgs.mkShell ({ - packages = with pkgs; [ rustc cargo rustfmt clippy ]; + packages = with pkgs; [ rustc cargo rustfmt clippy python3 ]; } // pkgs.lib.optionalAttrs pkgs.stdenv.hostPlatform.isDarwin { # minifb 0.28 hard-codes macOS 10.10 before user CFLAGS. # Match the Nix SDK's deployment target so Metal APIs are available. diff --git a/tools/gba-tests-arm.lock.json b/tools/gba-tests-arm.lock.json new file mode 100644 index 0000000..089c685 --- /dev/null +++ b/tools/gba-tests-arm.lock.json @@ -0,0 +1,91 @@ +{ + "repository": "jsmolka/gba-tests", + "revision": "a7113b67e63f83a9b321696ddd7042ccfad6c881", + "license": "MIT", + "rom": "arm/arm.gba", + "completion_pc": "0x08001d4c", + "result_register": 12, + "files": { + "LICENSE": { + "sha256": "b59a9ed235d76752c977f04116bc72155a14af48176bdbf50d7e5c29fca35aa7", + "bytes": 1070 + }, + "README.md": { + "sha256": "488f50d9d9031c88cea4161fad69f6460209dd54d965645fa45cf5e972cc9062", + "bytes": 434 + }, + "arm/arm.asm": { + "sha256": "eb708021f0bc77e022516e5770862c267a54513b45a4b0c81eae921b4a582283", + "bytes": 1070 + }, + "arm/arm.gba": { + "sha256": "77ee88662552bdc885c1080c0172ff119d54db791bd73b21808cf1ff1fe5b40e", + "bytes": 8824 + }, + "arm/block_transfer.asm": { + "sha256": "0e6483405a80b6d36b6ac6bbe0543fe42822cc42bc288b96f64b45aae1676ccf", + "bytes": 9888 + }, + "arm/branches.asm": { + "sha256": "92ac6cd8ff9532c2cb0e5ccff183558e0ae5bc63eee6e2e0ad86af3ec0138786", + "bytes": 910 + }, + "arm/conditions.asm": { + "sha256": "3f4c5835bca66647d45b2f7573bab80b64fb132334c67f97a11e892efefd1a62", + "bytes": 1992 + }, + "arm/data_processing.asm": { + "sha256": "19fa7c26807bf83dababe46c76b120bc0f20adbb6c7fbf60fd900c6391698a0d", + "bytes": 8712 + }, + "arm/data_swap.asm": { + "sha256": "f7280d42f8a8533143b90ca1ffd5aa5cc689cc2dadd60010278e602ecf1ecb8e", + "bytes": 1474 + }, + "arm/flags.asm": { + "sha256": "99721b1054668061c1287ff93479829bdd2ebffb706f6aed9200270457434f46", + "bytes": 2639 + }, + "arm/halfword_transfer.asm": { + "sha256": "79a6e93c07c68ad1c8d23c72f08e6ddca7aac805a496770169fc859a34e8177f", + "bytes": 4293 + }, + "arm/multiply.asm": { + "sha256": "8d751321327ba6104e94c1804f8976dbe126b14025d72a5b2f58a08facf2694a", + "bytes": 4803 + }, + "arm/psr_transfer.asm": { + "sha256": "d60520c2c3b104a009881bdc597b255af460a667b6fccf801a24aab5986b453c", + "bytes": 1431 + }, + "arm/shifts.asm": { + "sha256": "ebfb709bbb4e21273f067f510b8cb536a5df67701599fecbca10960ffcb2b8f0", + "bytes": 4473 + }, + "arm/single_transfer.asm": { + "sha256": "b2ce52f89735c5571a41d0561f44fb908e927357a6e6228acc24119e21fc2a90", + "bytes": 3863 + }, + "lib/constants.inc": { + "sha256": "b4dd3f4760ac1ae7e99cdc7e255f8220eed666e1ab821b150561db58e5d3b7ea", + "bytes": 900 + }, + "lib/macros.inc": { + "sha256": "25b0ff553c4ca99d76ec82c8c9db874e25d6eed58e2e69fa27f1a0fe7a8c46d4", + "bytes": 2864 + }, + "lib/header.asm": { + "sha256": "e9d1e8c7b6382eeb1b7059423ec365e32911f702eadfa51804a2119438a87357", + "bytes": 1720 + }, + "lib/text.asm": { + "sha256": "77e579d6cf0db182fd24e8d420c8c0a81592d78197e8fb06accc8bcfb43db17e", + "bytes": 2288 + }, + "lib/glyphs.asm": { + "sha256": "538beb37f40b1fd56beed9253e0e9eab25444c748adba7440a2093f49e3cbcc9", + "bytes": 2545 + } + } +} + diff --git a/tools/prepare_gba_tests.py b/tools/prepare_gba_tests.py new file mode 100644 index 0000000..3a9c9ff --- /dev/null +++ b/tools/prepare_gba_tests.py @@ -0,0 +1,112 @@ +#!/usr/bin/env python3 +"""Prepare a hash-verified, pinned public ARM test outside tracked source paths. + +This adapter is original. Upstream sources, license, and ROM bytes are downloaded +only to the requested local directory. They are not embedded in this repository. +""" + +import argparse +import hashlib +import json +from pathlib import Path +import struct +import sys +from urllib.request import Request, urlopen + +ROOT = Path(__file__).resolve().parent.parent +LOCK = Path(__file__).with_name("gba-tests-arm.lock.json") +MAX_FILE_BYTES = 1024 * 1024 + + +def fetch(url): + request = Request(url, headers={"User-Agent": "gameboy-rust-public-test-adapter"}) + with urlopen(request, timeout=30) as response: + return response.read(MAX_FILE_BYTES + 1) + + +def verify(name, data, expected): + if len(data) > MAX_FILE_BYTES or len(data) != expected["bytes"]: + raise ValueError(f"size mismatch for {name}") + if hashlib.sha256(data).hexdigest() != expected["sha256"]: + raise ValueError(f"SHA-256 mismatch for {name}") + + +def destination(path): + path = Path(path).resolve() + # Public ROM headers can contain third-party assets. Keep them untracked. + if path.is_relative_to(ROOT) and not path.is_relative_to(ROOT / "roms"): + raise ValueError("output must be outside the repository or inside ignored roms/") + if path.exists(): + raise ValueError(f"output directory already exists: {path}") + if not path.parent.is_dir(): + raise ValueError("output parent directory must already exist") + return path + + +def prepare(output, source=None, download=fetch): + output = destination(output) + lock = json.loads(LOCK.read_text(encoding="utf-8")) + base = f"https://raw.githubusercontent.com/{lock['repository']}/{lock['revision']}/" + files = {} + for name, expected in lock["files"].items(): + if source is None: + data = download(base + name) + else: + path = Path(source) / name + if not path.is_file() or path.stat().st_size > MAX_FILE_BYTES: + raise ValueError(f"source must be a bounded regular file: {name}") + with path.open("rb") as stream: + data = stream.read(MAX_FILE_BYTES + 1) + verify(name, data, expected) + files[name] = data + + # The source-defined eval label starts with m_vsync's register save. All + # failure exits branch here with r12 set; successful fallthrough keeps r12=0. + # Hash pinning is authoritative; this check also catches adapter-offset mistakes. + pc = int(lock["completion_pc"], 16) + offset = pc - 0x08000000 + rom = files[lock["rom"]] + if offset < 0 or offset + 4 > len(rom) or struct.unpack_from("