test(cpu): integrate pinned public Thumb compatibility suite
Add --suite thumb while retaining default ARM preparation. Verify hashes, the Thumb-to-ARM evaluation bridge, and the result checkpoint; retain upstream provenance outside tracked files. Expand offline synthetic adapter and CLI regressions. Darwin arm64 baseline: the unmodified Thumb ROM reaches ARM eval at 0x08000934 after 771 steps with r7=229 in debug and release. ARM remains passing. Online/offline preparation agrees and all 21 Python tests pass. CPU behavior is unchanged.
This commit is contained in:
0xrsydn
parent
a9493d4cc6
commit
08c8705904
11 changed files with 304 additions and 50 deletions
|
|
@ -34,7 +34,7 @@ Window smoke tests (`--*-smoke-test`) need a logged-in desktop session. Details
|
|||
| Working on | Read |
|
||||
| --- | --- |
|
||||
| ARM/Thumb instructions, modes, exceptions, timing | [docs/hardware/cpu.md](docs/hardware/cpu.md) |
|
||||
| Pinned public ARM tests, results, and compatibility limits | [docs/public-arm-tests.md](docs/public-arm-tests.md) |
|
||||
| Pinned public ARM/Thumb tests and compatibility results | [docs/public-arm-tests.md](docs/public-arm-tests.md), [docs/public-thumb-tests.md](docs/public-thumb-tests.md) |
|
||||
| ROM test suites, assertions, and JSON reports | [docs/rom-tests.md](docs/rom-tests.md) |
|
||||
| ROM files, window execution, and terminal diagnostics | [docs/hardware/cartridge.md](docs/hardware/cartridge.md) |
|
||||
| BIOS services (SWI) | [docs/hardware/bios.md](docs/hardware/bios.md) |
|
||||
|
|
|
|||
|
|
@ -42,6 +42,7 @@ Use `--test-suite suite.json` for [repeatable headless ROM assertions](docs/rom-
|
|||
| [docs/testing.md](docs/testing.md) | Validation commands and what the test suites cover |
|
||||
| [docs/rom-tests.md](docs/rom-tests.md) | Headless ROM suites, completion checks, assertions, and JSON reports |
|
||||
| [docs/public-arm-tests.md](docs/public-arm-tests.md) | Pinned public ARM test, preparation command, results, and limits |
|
||||
| [docs/public-thumb-tests.md](docs/public-thumb-tests.md) | Pinned public Thumb test, result protocol, and compatibility baseline |
|
||||
| [docs/architecture.md](docs/architecture.md) | Crate boundaries and a file-by-file map |
|
||||
| [docs/hardware/cpu.md](docs/hardware/cpu.md) | ARM/Thumb execution rules, modes, exceptions, and timing |
|
||||
| [docs/hardware/cartridge.md](docs/hardware/cartridge.md) | ROM loading, window/input controls, original ROM generator, and diagnostics |
|
||||
|
|
|
|||
|
|
@ -14,7 +14,7 @@ The repository is a Cargo workspace with three crates:
|
|||
| Path | Purpose |
|
||||
| --- | --- |
|
||||
| `flake.nix`, `flake.lock` | Pinned Rust/Python development environment |
|
||||
| `tools/prepare_gba_tests.py`, `tools/gba-tests-arm.lock.json` | Hash-verified public ARM source/ROM preparation and suite generation |
|
||||
| `tools/prepare_gba_tests.py`, `tools/gba-tests-{arm,thumb}.lock.json` | Hash-verified public ARM/Thumb source/ROM preparation and suite generation |
|
||||
| `tools/test_prepare_gba_tests.py` | Offline adapter tests using original synthetic data |
|
||||
| `.envrc` | Automatic environment loading through direnv |
|
||||
| `AGENTS.md` | Short orientation for coding agents: crate map, commands, and where docs live |
|
||||
|
|
|
|||
81
docs/public-thumb-tests.md
Normal file
81
docs/public-thumb-tests.md
Normal file
|
|
@ -0,0 +1,81 @@
|
|||
# Pinned public Thumb test
|
||||
|
||||
The headless runner executes the unmodified Thumb ROM from [jsmolka/gba-tests](https://github.com/jsmolka/gba-tests).
|
||||
It currently reaches its result checkpoint but fails test 229 on Darwin arm64 in debug and release builds.
|
||||
This is an independent compatibility baseline, not complete ARM7TDMI conformance or a hardware measurement.
|
||||
|
||||
## Prepare and run
|
||||
|
||||
```sh
|
||||
direnv exec . python3 tools/prepare_gba_tests.py /tmp/gba-public-thumb --suite thumb
|
||||
direnv exec . cargo run --locked --release -- --test-suite /tmp/gba-public-thumb/suite.json > /tmp/gba-public-thumb-report.json
|
||||
```
|
||||
|
||||
Use a new directory outside the repository or inside ignored `roms/`. The parent directory must exist.
|
||||
The adapter checks every selected file's byte count and SHA-256 digest before creating output.
|
||||
It retains the upstream sources, license, README, ROM, generated `suite.json`, and `source-lock.json`.
|
||||
No source, binary, header, expected result, or instruction is patched or skipped.
|
||||
Downloaded binaries and header sources must not enter tracked files.
|
||||
|
||||
Offline preparation uses the same checks:
|
||||
|
||||
```sh
|
||||
direnv exec . python3 tools/prepare_gba_tests.py /tmp/gba-public-thumb-offline --suite thumb --source-dir /path/to/gba-tests
|
||||
```
|
||||
|
||||
The existing [ARM command](public-arm-tests.md) remains the default. Explicit `--suite arm` selects the same ARM lock.
|
||||
Each preparation creates one suite. The suite name does not select the checkpoint's instruction state.
|
||||
Python comes from Nix and requires no extra packages. Ordinary unit tests use original synthetic data, without downloads.
|
||||
Preparation is not a compatibility verdict. Inspect the runner's JSON result and exit status.
|
||||
|
||||
## Provenance and result protocol
|
||||
|
||||
- Repository: `jsmolka/gba-tests`.
|
||||
- Revision: `a7113b67e63f83a9b321696ddd7042ccfad6c881`.
|
||||
- Upstream project license: MIT, copyright Julian Smolka; retained with downloaded files.
|
||||
- ROM: `thumb/thumb.gba`, 3,680 bytes.
|
||||
- SHA-256: `b5cb2291df4ab314b31c598acd9bff2ccfa0b38efff29daadfe97422ce369b67`.
|
||||
- Selected file hashes and sizes: [`tools/gba-tests-thumb.lock.json`](../tools/gba-tests-thumb.lock.json).
|
||||
|
||||
The adapter uses the upstream committed binary. No FASMARM rebuild or source-to-binary reproduction was performed on Darwin.
|
||||
The entry code initializes the display, switches to Thumb, and initializes r7 to zero.
|
||||
Each `m_exit` failure writes its test number to r7 and calls `tmain_end`.
|
||||
Branch tests also use r7 as a progress marker, then reset it to zero on their successful path.
|
||||
Successful completion of the memory tests reaches the same `tmain_end` with r7 still zero.
|
||||
|
||||
At this revision:
|
||||
|
||||
- `tmain_end` is Thumb code at `0x08000930`.
|
||||
- `ADR r0, eval` (`0xa000`) and `BX r0` (`0x4700`) switch execution back to ARM.
|
||||
- `eval` is ARM code at `0x08000934`, starting with `0xe92d0003`, the `m_vsync` register save.
|
||||
- The later result macro reads r7, not the ARM suite's r12.
|
||||
|
||||
The source exit paths and binary Thumb `BL` targets were checked against `tmain_end`.
|
||||
The adapter verifies the bridge and initial evaluation word, in addition to whole-file hashes.
|
||||
The manifest stops before `eval`, requires ARM state and `r7 == 0`, and allows 1,000,000 successful machine steps.
|
||||
The budget includes the original BIOS boot. Startup and CPU tests execute normally; result rendering does not execute.
|
||||
Diagnostics, timeouts, STOP, and nonzero r7 remain failures under the [runner's normal rules](rom-tests.md).
|
||||
|
||||
## Observed result on Darwin arm64
|
||||
|
||||
Initial integration, before fixing empty-list Thumb stores:
|
||||
|
||||
- The runner reaches `0x08000934` after 771 successful steps and 3,858 nominal cycles.
|
||||
- The CPU is in System/ARM state with CPSR `0x0000001f` and r7 = 229.
|
||||
- Both builds exit nonzero with `reason: assertion_failed`; there is no emulator diagnostic.
|
||||
- Test 229 executes `STMIA r0!, {}` (`0xc000`) at `0x080008ba`.
|
||||
- The stored PC value is `0x080008be`, but the following PC read expects `0x080008c0`.
|
||||
- The public ARM ROM still passes its unchanged checkpoint.
|
||||
|
||||
The next CPU task is to verify the empty-list store PC offset and correct it with original regressions.
|
||||
Do not change the pinned ROM or weaken the result assertion to pass this test.
|
||||
No public timing, memory, BIOS, graphics, or unsafe suite is claimed as passing.
|
||||
These results do not establish Pokémon Emerald compatibility.
|
||||
|
||||
## References
|
||||
|
||||
- [Pinned Thumb entry and result flow](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/thumb.asm).
|
||||
- [Branch progress markers and successful reset](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/branches.asm).
|
||||
- [Memory tests, including test 229](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/memory.asm).
|
||||
- [Shared startup and result macros](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/lib/macros.inc).
|
||||
- [Pinned license](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/LICENSE).
|
||||
|
|
@ -65,6 +65,7 @@ Hardware references used for the core:
|
|||
- [GBATEK ARM data processing](https://problemkaputt.de/gbatek-arm-opcodes-data-processing-alu.htm), including R15/status notes. Its legacy `{P}` description alone does not establish ARM7 behavior.
|
||||
- [mGBA ARM instruction implementation](https://github.com/mgba-emu/mgba/blob/master/src/arm/isa-arm.c), reviewed for Rd=15 test/compare SPSR restoration, User/System fallback, and no result write/refill. Also reviewed its word/byte/halfword/signed load ordering: writeback precedes the destination result.
|
||||
- [jsmolka/gba-tests](https://github.com/jsmolka/gba-tests/tree/a7113b67e63f83a9b321696ddd7042ccfad6c881), pinned for independent ARM testing. Reviewed its MIT license, startup, result macros, compare/status tests, and load/writeback alias tests.
|
||||
- [Pinned jsmolka Thumb entry](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/thumb.asm), [branch tests](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/branches.asm), and [memory tests](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/memory.asm), reviewed for r7 failure IDs, success-marker reset, the Thumb-to-ARM evaluation bridge, and empty-list transfer expectations.
|
||||
- [armwrestler-gba-fixed](https://github.com/destoer/armwrestler-gba-fixed) and [arm7wrestler](https://github.com/Arisotura/arm7wrestler), considered as test sources; not integrated.
|
||||
- [SingleStepTests ARM7TDMI](https://github.com/SingleStepTests/ARM7TDMI), considered as an experimental, emulator-generated test source; not integrated.
|
||||
- [GBATEK ARM single data transfers](https://problemkaputt.de/gbatek-arm-opcodes-memory-single-data-transfer-ldr-str-pld.htm), for address calculation, indexing, load extension, and nominal timing.
|
||||
|
|
|
|||
|
|
@ -4,8 +4,9 @@ Use `--test-suite PATH.json` to run repeatable ROM checks without opening a wind
|
|||
Each case has a machine-step budget, a completion address, and explicit assertions.
|
||||
A timeout never counts as a pass. Existing `--rom ... --steps ...` remains a diagnostic mode, not a test verdict.
|
||||
|
||||
Original programs cover the runner itself. A [pinned public ARM test](public-arm-tests.md) now provides an independent compatibility baseline.
|
||||
That public ROM still fails; passing the runner's unit tests does not mean the external ARM suite passes.
|
||||
Original programs cover the runner itself. Pinned public [ARM](public-arm-tests.md) and [Thumb](public-thumb-tests.md) ROMs provide independent compatibility results.
|
||||
The ARM ROM passes its checkpoint. The Thumb ROM currently reports test 229 as failed.
|
||||
Passing the runner's unit tests does not mean an external suite passes.
|
||||
A passing suite establishes only its specified checkpoints and assertions. It does not establish Emerald compatibility.
|
||||
|
||||
## Run the original smoke suite
|
||||
|
|
@ -132,7 +133,8 @@ They do not hash ROM contents; record source revisions and file checksums alongs
|
|||
|
||||
The first public integration is [jsmolka/gba-tests ARM](public-arm-tests.md), with a verified result-register checkpoint.
|
||||
The pinned ARM ROM now passes this checkpoint on Darwin arm64. Historical failures remain documented.
|
||||
Next, integrate a pinned public Thumb suite with a verified result protocol.
|
||||
The [pinned Thumb ROM](public-thumb-tests.md) also has a verified checkpoint, but currently fails its r7 assertion at test 229.
|
||||
Verify and correct the empty-list store PC value before claiming that suite passes.
|
||||
For further public ARM7TDMI/GBA tests, inspect their source, license, entry assumptions, and result protocol.
|
||||
Pin source revisions and build instructions before comparing emulator changes.
|
||||
Adapt suites to verified completion addresses and result locations; do not guess them or treat timeouts as passes.
|
||||
|
|
|
|||
|
|
@ -67,7 +67,8 @@ Update this file when a feature lands or a limit is removed.
|
|||
- ROM windows with scanline capture, keyboard input, focus-loss release, bounded execution slices, and interactive keypad wake from STOP.
|
||||
- Optional ROM-window frame limits and an original input-test ROM generator.
|
||||
- Headless JSON ROM suites with per-case checkpoints, register/CPSR/memory assertions, bounded budgets, and structured failure reports.
|
||||
- Hash-pinned preparation of the public `jsmolka/gba-tests` ARM ROM, passing its result checkpoint on Darwin arm64.
|
||||
- Hash-pinned preparation of public `jsmolka/gba-tests` ARM and Thumb ROMs, with verified result-register checkpoints.
|
||||
- The public ARM ROM passes on Darwin arm64; the Thumb ROM has an explicit failing baseline at test 229.
|
||||
- ARM test/compare Rd=15 status restoration, without a PC result write or nominal refill.
|
||||
- ARM unused-memory data reads return a PC+8 word snapshot, with normal byte lanes and load rotation.
|
||||
- Little-endian byte, halfword, and word reads/writes.
|
||||
|
|
@ -115,7 +116,8 @@ ROM windows have no demo-specific startup checks. Input is sampled between bound
|
|||
Audio, saves, and external BIOS loading remain unavailable. ROM loading does not establish commercial-game compatibility.
|
||||
See [cartridge loading](hardware/cartridge.md) for file limits, step budgets, and exit status.
|
||||
The [ROM suite runner](rom-tests.md) has original regressions and a [pinned public ARM baseline](public-arm-tests.md).
|
||||
The pinned public ARM ROM passes its result checkpoint in debug and release builds. Other public suites remain unverified.
|
||||
The pinned public ARM ROM passes its result checkpoint in debug and release builds.
|
||||
The [pinned public Thumb ROM](public-thumb-tests.md) reaches its checkpoint but reports test 229 as failed. Other public suites remain unverified.
|
||||
The runner requires known completion addresses and has no debug-port protocol, scripted input, or rendered-image assertions.
|
||||
|
||||
ARM data reads from unused address ranges use a bounded [PC+8 open-bus snapshot](hardware/cpu.md#arm-unused-memory-data-reads).
|
||||
|
|
@ -128,7 +130,7 @@ Writes to cartridge addresses return errors instead of modeling cartridge hardwa
|
|||
|
||||
## Next steps
|
||||
|
||||
1. Add a pinned public Thumb suite with a verified result protocol, then address its first compatibility failure.
|
||||
1. Verify and fix the empty-list Thumb store PC value exposed by public test 229, then rerun both pinned suites.
|
||||
Extend open-bus support with verified Thumb region/alignment rules and bus history, rather than a blanket unmapped-read fallback.
|
||||
2. Refine nominal timing with a fetch pipeline, Game Pak prefetch, per-access device updates, and verified timer/IRQ delays.
|
||||
3. Validate keypad retrigger behavior with hardware tests and add remaining device registers; extend BIOS reset coverage as sound/serial support becomes available.
|
||||
|
|
|
|||
|
|
@ -50,13 +50,15 @@ The [headless ROM suite runner](rom-tests.md) provides explicit checkpoint and a
|
|||
Its tests verify manifest limits, whole-suite validation, exact step boundaries, ARM/Thumb checkpoints, and register/CPSR/memory checks.
|
||||
Additional tests cover DMA/IRQ accounting, HALT/STOP, diagnostic state, case isolation, repeatable JSON output, and failure exit status.
|
||||
These original tests validate the runner. The [public ARM adapter](public-arm-tests.md) provides a separate passing checkpoint result for its pinned ROM.
|
||||
Its preparation tests use only synthetic original bytes and no network:
|
||||
The [public Thumb adapter](public-thumb-tests.md) uses a separate lock and r7 assertion after its return to ARM state.
|
||||
Preparation tests use only synthetic original bytes and no network:
|
||||
|
||||
```sh
|
||||
direnv exec . python3 -B -m unittest discover -s tools -p 'test_*.py' -v
|
||||
```
|
||||
|
||||
They verify pinned URLs, hashes, sizes, checkpoint validation, offline copies, destination safety, and non-overwriting output.
|
||||
They verify suite selection, pinned URLs, hashes, sizes, checkpoint validation, offline copies, destination safety, and non-overwriting output.
|
||||
Thumb cases also verify the return-to-ARM bridge. Command-line tests cover default ARM selection, explicit Thumb selection, and errors.
|
||||
Compare/status CPU regressions verify saved flags/masks, bank switching, sequential PC/timing, User/System fallback, and atomic diagnostics.
|
||||
Load-alias regressions cover widths, signed/unaligned values, immediate/register offsets, pre/post-indexing, every non-PC register bank, and address wrapping.
|
||||
They verify loaded-value precedence, preserved flags, unchanged stores, nominal timing, I/O reads before device progress, deferred IRQ delivery, and atomic errors.
|
||||
|
|
|
|||
66
tools/gba-tests-thumb.lock.json
Normal file
66
tools/gba-tests-thumb.lock.json
Normal file
|
|
@ -0,0 +1,66 @@
|
|||
{
|
||||
"repository": "jsmolka/gba-tests",
|
||||
"revision": "a7113b67e63f83a9b321696ddd7042ccfad6c881",
|
||||
"license": "MIT",
|
||||
"rom": "thumb/thumb.gba",
|
||||
"completion_pc": "0x08000934",
|
||||
"result_register": 7,
|
||||
"files": {
|
||||
"LICENSE": {
|
||||
"sha256": "b59a9ed235d76752c977f04116bc72155a14af48176bdbf50d7e5c29fca35aa7",
|
||||
"bytes": 1070
|
||||
},
|
||||
"README.md": {
|
||||
"sha256": "488f50d9d9031c88cea4161fad69f6460209dd54d965645fa45cf5e972cc9062",
|
||||
"bytes": 434
|
||||
},
|
||||
"thumb/thumb.asm": {
|
||||
"sha256": "090a6bb08b3813e2da9adef9705368d5c292bd563aeada82189f93b5b16e0f6e",
|
||||
"bytes": 790
|
||||
},
|
||||
"thumb/thumb.gba": {
|
||||
"sha256": "b5cb2291df4ab314b31c598acd9bff2ccfa0b38efff29daadfe97422ce369b67",
|
||||
"bytes": 3680
|
||||
},
|
||||
"thumb/arithmetic.asm": {
|
||||
"sha256": "3da934b2b05f208c6ddea8617fa281845db09816b4e4c050990a2e0c0c82724f",
|
||||
"bytes": 4990
|
||||
},
|
||||
"thumb/branches.asm": {
|
||||
"sha256": "69c6ce873b68beebcf1cea49a057ebc360d1539c8c112d607563757ed7414f60",
|
||||
"bytes": 1766
|
||||
},
|
||||
"thumb/logical.asm": {
|
||||
"sha256": "f3c70c79867ec57f0826f727246650ba12d1156085eb4e0678deed5177343f46",
|
||||
"bytes": 2436
|
||||
},
|
||||
"thumb/memory.asm": {
|
||||
"sha256": "250b5b0e4a616f2b836d8fe525df4092936a5992f8b2143c7784cdbe42a84e03",
|
||||
"bytes": 10357
|
||||
},
|
||||
"thumb/shifts.asm": {
|
||||
"sha256": "cc7c8395c5e9abbb220f47af507d9256378a8186a0514b4c73add1dc12db5e70",
|
||||
"bytes": 5018
|
||||
},
|
||||
"lib/constants.inc": {
|
||||
"sha256": "b4dd3f4760ac1ae7e99cdc7e255f8220eed666e1ab821b150561db58e5d3b7ea",
|
||||
"bytes": 900
|
||||
},
|
||||
"lib/macros.inc": {
|
||||
"sha256": "25b0ff553c4ca99d76ec82c8c9db874e25d6eed58e2e69fa27f1a0fe7a8c46d4",
|
||||
"bytes": 2864
|
||||
},
|
||||
"lib/header.asm": {
|
||||
"sha256": "e9d1e8c7b6382eeb1b7059423ec365e32911f702eadfa51804a2119438a87357",
|
||||
"bytes": 1720
|
||||
},
|
||||
"lib/text.asm": {
|
||||
"sha256": "77e579d6cf0db182fd24e8d420c8c0a81592d78197e8fb06accc8bcfb43db17e",
|
||||
"bytes": 2288
|
||||
},
|
||||
"lib/glyphs.asm": {
|
||||
"sha256": "538beb37f40b1fd56beed9253e0e9eab25444c748adba7440a2093f49e3cbcc9",
|
||||
"bytes": 2545
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Prepare a hash-verified, pinned public ARM test outside tracked source paths.
|
||||
"""Prepare a hash-verified, pinned public ARM or Thumb test outside tracked paths.
|
||||
|
||||
This adapter is original. Upstream sources, license, and ROM bytes are downloaded
|
||||
only to the requested local directory. They are not embedded in this repository.
|
||||
|
|
@ -14,7 +14,8 @@ import sys
|
|||
from urllib.request import Request, urlopen
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
LOCK = Path(__file__).with_name("gba-tests-arm.lock.json")
|
||||
LOCKS = {suite: Path(__file__).with_name(f"gba-tests-{suite}.lock.json")
|
||||
for suite in ("arm", "thumb")}
|
||||
MAX_FILE_BYTES = 1024 * 1024
|
||||
|
||||
|
||||
|
|
@ -43,9 +44,11 @@ def destination(path):
|
|||
return path
|
||||
|
||||
|
||||
def prepare(output, source=None, download=fetch):
|
||||
def prepare(output, source=None, download=fetch, suite="arm"):
|
||||
if suite not in LOCKS:
|
||||
raise ValueError(f"unsupported public suite: {suite}")
|
||||
output = destination(output)
|
||||
lock = json.loads(LOCK.read_text(encoding="utf-8"))
|
||||
lock = json.loads(LOCKS[suite].read_text(encoding="utf-8"))
|
||||
base = f"https://raw.githubusercontent.com/{lock['repository']}/{lock['revision']}/"
|
||||
files = {}
|
||||
for name, expected in lock["files"].items():
|
||||
|
|
@ -61,17 +64,20 @@ def prepare(output, source=None, download=fetch):
|
|||
files[name] = data
|
||||
|
||||
# The source-defined eval label starts with m_vsync's register save. All
|
||||
# failure exits branch here with r12 set; successful fallthrough keeps r12=0.
|
||||
# Hash pinning is authoritative; this check also catches adapter-offset mistakes.
|
||||
# failure exits arrive with the result register set (ARM r12, Thumb r7).
|
||||
# Thumb tests switch back to ARM before eval. Success leaves the result zero.
|
||||
# Hash pinning is authoritative; these checks catch adapter-offset mistakes.
|
||||
pc = int(lock["completion_pc"], 16)
|
||||
offset = pc - 0x08000000
|
||||
rom = files[lock["rom"]]
|
||||
if offset < 0 or offset + 4 > len(rom) or struct.unpack_from("<I", rom, offset)[0] != 0xE92D0003:
|
||||
if offset < 0 or offset % 4 or offset + 4 > len(rom) or struct.unpack_from("<I", rom, offset)[0] != 0xE92D0003:
|
||||
raise ValueError("pinned evaluation checkpoint does not match the ROM")
|
||||
if suite == "thumb" and (offset < 4 or struct.unpack_from("<HH", rom, offset - 4) != (0xA000, 0x4700)):
|
||||
raise ValueError("pinned Thumb-to-ARM checkpoint bridge does not match the ROM")
|
||||
manifest = {
|
||||
"version": 1,
|
||||
"cases": [{
|
||||
"name": "jsmolka-arm",
|
||||
"name": f"jsmolka-{suite}",
|
||||
"rom": lock["rom"],
|
||||
"step_limit": 1_000_000,
|
||||
"completion": {"pc": lock["completion_pc"], "instruction_set": "arm"},
|
||||
|
|
@ -96,14 +102,15 @@ def prepare(output, source=None, download=fetch):
|
|||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("output", type=Path, help="new directory outside the repository or under roms/")
|
||||
parser.add_argument("--suite", choices=tuple(LOCKS), default="arm", help="public suite to prepare (default: arm)")
|
||||
parser.add_argument("--source-dir", type=Path, help="offline upstream tree; every selected file must match the lock")
|
||||
args = parser.parse_args()
|
||||
try:
|
||||
manifest = prepare(args.output, args.source_dir)
|
||||
manifest = prepare(args.output, args.source_dir, suite=args.suite)
|
||||
except (OSError, ValueError) as error:
|
||||
print(f"cannot prepare public ARM test: {error}", file=sys.stderr)
|
||||
print(f"cannot prepare public {args.suite} test: {error}", file=sys.stderr)
|
||||
return 1
|
||||
print(f"Prepared pinned public ARM suite: {manifest}")
|
||||
print(f"Prepared pinned public {args.suite} suite: {manifest}")
|
||||
print("Run with gameboy-rust --test-suite PATH. Preparation does not mean the CPU tests passed.")
|
||||
return 0
|
||||
|
||||
|
|
|
|||
|
|
@ -1,5 +1,6 @@
|
|||
"""Adapter regressions with original synthetic bytes; no network or public ROMs."""
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
from pathlib import Path
|
||||
import struct
|
||||
|
|
@ -11,29 +12,41 @@ import prepare_gba_tests as adapter
|
|||
|
||||
|
||||
class PreparationTests(unittest.TestCase):
|
||||
suite = "arm"
|
||||
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.root = Path(self.temp.name).resolve()
|
||||
self.output = self.root / "prepared"
|
||||
self.rom = f"{self.suite}/{self.suite}.gba"
|
||||
if self.suite == "arm":
|
||||
data = struct.pack("<II", 0xEAFFFFFE, 0xE92D0003)
|
||||
pc, result = "0x08000004", 12
|
||||
else:
|
||||
data = struct.pack("<IIHHI", 0xEAFFFFFE, 0xE1A00000, 0xA000, 0x4700, 0xE92D0003)
|
||||
pc, result = "0x0800000c", 7
|
||||
self.files = {
|
||||
"arm/arm.gba": struct.pack("<II", 0xEAFFFFFE, 0xE92D0003),
|
||||
self.rom: data,
|
||||
"LICENSE": b"Original fixture, not upstream license content.\n",
|
||||
"arm/arm.asm": b"Original fixture source marker.\n",
|
||||
f"{self.suite}/{self.suite}.asm": b"Original fixture source marker.\n",
|
||||
}
|
||||
self.lock = {
|
||||
"repository": "original/test-fixture",
|
||||
"revision": "0" * 40,
|
||||
"license": "fixture",
|
||||
"rom": "arm/arm.gba",
|
||||
"completion_pc": "0x08000004",
|
||||
"result_register": 12,
|
||||
"rom": self.rom,
|
||||
"completion_pc": pc,
|
||||
"result_register": result,
|
||||
"files": {name: {"bytes": len(data), "sha256": hashlib.sha256(data).hexdigest()}
|
||||
for name, data in self.files.items()},
|
||||
}
|
||||
self.lock_path = self.root / "lock.json"
|
||||
self.save_lock()
|
||||
patch = mock.patch.object(adapter, "LOCK", self.lock_path)
|
||||
# The unselected path does not exist. Reading the wrong lock must fail.
|
||||
locks = {suite: self.root / f"missing-{suite}.json" for suite in adapter.LOCKS}
|
||||
locks[self.suite] = self.lock_path
|
||||
patch = mock.patch.object(adapter, "LOCKS", locks)
|
||||
patch.start()
|
||||
self.addCleanup(patch.stop)
|
||||
|
||||
|
|
@ -46,7 +59,17 @@ class PreparationTests(unittest.TestCase):
|
|||
return self.files[url[len(prefix):]]
|
||||
|
||||
def prepare(self):
|
||||
return adapter.prepare(self.output, download=self.download)
|
||||
# Exercise the existing default ARM API as well as explicit Thumb selection.
|
||||
options = {} if self.suite == "arm" else {"suite": self.suite}
|
||||
return adapter.prepare(self.output, download=self.download, **options)
|
||||
|
||||
def offline_source(self):
|
||||
source = self.root / "source"
|
||||
for name, data in self.files.items():
|
||||
target = source / name
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_bytes(data)
|
||||
return source
|
||||
|
||||
def test_verified_bytes_license_and_provenance_are_retained(self):
|
||||
manifest_path = self.prepare()
|
||||
|
|
@ -54,46 +77,50 @@ class PreparationTests(unittest.TestCase):
|
|||
self.assertEqual((self.output / name).read_bytes(), expected)
|
||||
self.assertEqual(json.loads((self.output / "source-lock.json").read_text()), self.lock)
|
||||
manifest = json.loads(manifest_path.read_text())
|
||||
case = manifest["cases"][0]
|
||||
self.assertEqual(case["completion"], {"pc": "0x08000004", "instruction_set": "arm"})
|
||||
self.assertEqual(case["checks"], [{"kind": "register", "index": 12, "equals": 0}])
|
||||
self.assertEqual(case["rom"], "arm/arm.gba")
|
||||
self.assertEqual(case["step_limit"], 1_000_000)
|
||||
self.assertEqual(manifest, {
|
||||
"version": 1,
|
||||
"cases": [{
|
||||
"name": f"jsmolka-{self.suite}",
|
||||
"rom": self.rom,
|
||||
"step_limit": 1_000_000,
|
||||
# The Thumb ROM returns to ARM before evaluation.
|
||||
"completion": {"pc": self.lock["completion_pc"], "instruction_set": "arm"},
|
||||
"checks": [{"kind": "register", "index": self.lock["result_register"], "equals": 0}],
|
||||
}],
|
||||
})
|
||||
|
||||
def test_existing_directory_is_rejected_before_fetching(self):
|
||||
self.output.mkdir()
|
||||
sentinel = self.output / "keep"
|
||||
sentinel.write_text("unchanged")
|
||||
with self.assertRaisesRegex(ValueError, "already exists"):
|
||||
adapter.prepare(self.output, download=lambda _: self.fail("unexpected download"))
|
||||
adapter.prepare(self.output, download=lambda _: self.fail("unexpected download"), suite=self.suite)
|
||||
self.assertEqual(sentinel.read_text(), "unchanged")
|
||||
|
||||
def test_failed_hash_or_size_leaves_no_output(self):
|
||||
for replacement in [b"short", b"x" * len(self.files["arm/arm.gba"])]:
|
||||
for replacement in [b"short", b"x" * len(self.files[self.rom])]:
|
||||
with self.subTest(replacement=replacement):
|
||||
self.files["arm/arm.gba"] = replacement
|
||||
self.files[self.rom] = replacement
|
||||
with self.assertRaisesRegex(ValueError, "mismatch"):
|
||||
self.prepare()
|
||||
self.assertFalse(self.output.exists())
|
||||
|
||||
def test_bad_checkpoint_is_rejected_even_with_matching_hash(self):
|
||||
self.lock["completion_pc"] = "0x08000000"
|
||||
for pc in ["0x07fffffc", "0x08000000", "0x08000003", "0x08001000"]:
|
||||
with self.subTest(pc=pc):
|
||||
self.lock["completion_pc"] = pc
|
||||
self.save_lock()
|
||||
with self.assertRaisesRegex(ValueError, "checkpoint"):
|
||||
self.prepare()
|
||||
self.assertFalse(self.output.exists())
|
||||
|
||||
def test_offline_tree_gets_the_same_checks_and_no_network(self):
|
||||
source = self.root / "source"
|
||||
for name, data in self.files.items():
|
||||
target = source / name
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_bytes(data)
|
||||
adapter.prepare(self.output, source, download=lambda _: self.fail("unexpected network"))
|
||||
self.assertEqual((self.output / "arm/arm.gba").read_bytes(), self.files["arm/arm.gba"])
|
||||
source = self.offline_source()
|
||||
adapter.prepare(self.output, source, download=lambda _: self.fail("unexpected network"), suite=self.suite)
|
||||
self.assertEqual((self.output / self.rom).read_bytes(), self.files[self.rom])
|
||||
(source / "LICENSE").write_bytes(b"modified")
|
||||
with self.assertRaisesRegex(ValueError, "mismatch"):
|
||||
adapter.prepare(self.root / "second", source)
|
||||
adapter.prepare(self.root / "second", source, suite=self.suite)
|
||||
self.assertFalse((self.root / "second").exists())
|
||||
|
||||
def test_download_failure_does_not_publish_partial_fixture(self):
|
||||
|
|
@ -105,7 +132,7 @@ class PreparationTests(unittest.TestCase):
|
|||
raise OSError("network unavailable")
|
||||
return self.download(url)
|
||||
with self.assertRaisesRegex(OSError, "network unavailable"):
|
||||
adapter.prepare(self.output, download=download)
|
||||
adapter.prepare(self.output, download=download, suite=self.suite)
|
||||
self.assertFalse(self.output.exists())
|
||||
|
||||
def test_tracked_repository_paths_are_rejected(self):
|
||||
|
|
@ -123,6 +150,71 @@ class PreparationTests(unittest.TestCase):
|
|||
with self.assertRaisesRegex(ValueError, "size mismatch"):
|
||||
adapter.verify("fixture", data, expected)
|
||||
|
||||
def test_cli_prepares_offline_and_reports_errors_without_network(self):
|
||||
source = self.offline_source()
|
||||
argv = ["prepare_gba_tests.py", str(self.output), "--source-dir", str(source)]
|
||||
if self.suite != "arm":
|
||||
argv += ["--suite", self.suite]
|
||||
with mock.patch("sys.argv", argv), mock.patch("sys.stdout", new_callable=io.StringIO) as stdout:
|
||||
self.assertEqual(adapter.main(), 0)
|
||||
self.assertIn(f"public {self.suite} suite", stdout.getvalue())
|
||||
# Existing output is an error, not an overwrite or a successful preparation.
|
||||
with mock.patch("sys.argv", argv), mock.patch("sys.stderr", new_callable=io.StringIO) as stderr:
|
||||
self.assertEqual(adapter.main(), 1)
|
||||
self.assertIn("already exists", stderr.getvalue())
|
||||
|
||||
|
||||
class ThumbPreparationTests(PreparationTests):
|
||||
suite = "thumb"
|
||||
|
||||
def test_thumb_bridge_is_checked_even_with_matching_hash(self):
|
||||
original = self.files[self.rom]
|
||||
# Wrong ADR, wrong BX, or no bridge before an otherwise valid eval word.
|
||||
for data, pc in [
|
||||
(original[:8] + struct.pack("<HHI", 0xA001, 0x4700, 0xE92D0003), "0x0800000c"),
|
||||
(original[:8] + struct.pack("<HHI", 0xA000, 0x4708, 0xE92D0003), "0x0800000c"),
|
||||
(struct.pack("<I", 0xE92D0003), "0x08000000"),
|
||||
]:
|
||||
with self.subTest(data=data):
|
||||
self.files[self.rom] = data
|
||||
self.lock["files"][self.rom] = {"bytes": len(data), "sha256": hashlib.sha256(data).hexdigest()}
|
||||
self.lock["completion_pc"] = pc
|
||||
self.save_lock()
|
||||
with self.assertRaisesRegex(ValueError, "Thumb-to-ARM checkpoint bridge"):
|
||||
self.prepare()
|
||||
self.assertFalse(self.output.exists())
|
||||
|
||||
|
||||
class SuiteSelectionTests(unittest.TestCase):
|
||||
def test_unknown_suite_is_rejected_before_download_or_output(self):
|
||||
with tempfile.TemporaryDirectory() as temp:
|
||||
output = Path(temp) / "prepared"
|
||||
with self.assertRaisesRegex(ValueError, "unsupported public suite"):
|
||||
adapter.prepare(output, suite="../unknown", download=lambda _: self.fail("unexpected network"))
|
||||
self.assertFalse(output.exists())
|
||||
with mock.patch("sys.argv", ["prepare_gba_tests.py", str(output), "--suite", "unknown"]):
|
||||
with mock.patch("sys.stderr", new_callable=io.StringIO), self.assertRaises(SystemExit) as exit:
|
||||
adapter.main()
|
||||
self.assertEqual(exit.exception.code, 2)
|
||||
self.assertFalse(output.exists())
|
||||
|
||||
def test_committed_locks_keep_distinct_verified_protocols(self):
|
||||
for suite, pc, register in [("arm", "0x08001d4c", 12), ("thumb", "0x08000934", 7)]:
|
||||
lock = json.loads(adapter.LOCKS[suite].read_text())
|
||||
self.assertEqual(lock["repository"], "jsmolka/gba-tests")
|
||||
self.assertEqual(lock["revision"], "a7113b67e63f83a9b321696ddd7042ccfad6c881")
|
||||
self.assertEqual(lock["license"], "MIT")
|
||||
self.assertEqual(lock["completion_pc"], pc)
|
||||
self.assertEqual(lock["result_register"], register)
|
||||
self.assertEqual(lock["rom"], f"{suite}/{suite}.gba")
|
||||
self.assertIn(lock["rom"], lock["files"])
|
||||
self.assertIn("LICENSE", lock["files"])
|
||||
for name, metadata in lock["files"].items():
|
||||
self.assertFalse(Path(name).is_absolute())
|
||||
self.assertNotIn("..", Path(name).parts)
|
||||
self.assertRegex(metadata["sha256"], r"^[0-9a-f]{64}$")
|
||||
self.assertTrue(0 < metadata["bytes"] <= adapter.MAX_FILE_BYTES)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue