test(cpu): integrate pinned public Thumb compatibility suite

Add --suite thumb while retaining default ARM preparation. Verify hashes, the Thumb-to-ARM evaluation bridge, and the result checkpoint; retain upstream provenance outside tracked files. Expand offline synthetic adapter and CLI regressions.

Darwin arm64 baseline: the unmodified Thumb ROM reaches ARM eval at 0x08000934 after 771 steps with r7=229 in debug and release. ARM remains passing. Online/offline preparation agrees and all 21 Python tests pass. CPU behavior is unchanged.
This commit is contained in:
0xrsydn 2026-10-05 20:40:37 +07:00
commit 08c8705904
11 changed files with 304 additions and 50 deletions

View file

@ -14,7 +14,7 @@ The repository is a Cargo workspace with three crates:
| Path | Purpose |
| --- | --- |
| `flake.nix`, `flake.lock` | Pinned Rust/Python development environment |
| `tools/prepare_gba_tests.py`, `tools/gba-tests-arm.lock.json` | Hash-verified public ARM source/ROM preparation and suite generation |
| `tools/prepare_gba_tests.py`, `tools/gba-tests-{arm,thumb}.lock.json` | Hash-verified public ARM/Thumb source/ROM preparation and suite generation |
| `tools/test_prepare_gba_tests.py` | Offline adapter tests using original synthetic data |
| `.envrc` | Automatic environment loading through direnv |
| `AGENTS.md` | Short orientation for coding agents: crate map, commands, and where docs live |

View file

@ -0,0 +1,81 @@
# Pinned public Thumb test
The headless runner executes the unmodified Thumb ROM from [jsmolka/gba-tests](https://github.com/jsmolka/gba-tests).
It currently reaches its result checkpoint but fails test 229 on Darwin arm64 in debug and release builds.
This is an independent compatibility baseline, not complete ARM7TDMI conformance or a hardware measurement.
## Prepare and run
```sh
direnv exec . python3 tools/prepare_gba_tests.py /tmp/gba-public-thumb --suite thumb
direnv exec . cargo run --locked --release -- --test-suite /tmp/gba-public-thumb/suite.json > /tmp/gba-public-thumb-report.json
```
Use a new directory outside the repository or inside ignored `roms/`. The parent directory must exist.
The adapter checks every selected file's byte count and SHA-256 digest before creating output.
It retains the upstream sources, license, README, ROM, generated `suite.json`, and `source-lock.json`.
No source, binary, header, expected result, or instruction is patched or skipped.
Downloaded binaries and header sources must not enter tracked files.
Offline preparation uses the same checks:
```sh
direnv exec . python3 tools/prepare_gba_tests.py /tmp/gba-public-thumb-offline --suite thumb --source-dir /path/to/gba-tests
```
The existing [ARM command](public-arm-tests.md) remains the default. Explicit `--suite arm` selects the same ARM lock.
Each preparation creates one suite. The suite name does not select the checkpoint's instruction state.
Python comes from Nix and requires no extra packages. Ordinary unit tests use original synthetic data, without downloads.
Preparation is not a compatibility verdict. Inspect the runner's JSON result and exit status.
## Provenance and result protocol
- Repository: `jsmolka/gba-tests`.
- Revision: `a7113b67e63f83a9b321696ddd7042ccfad6c881`.
- Upstream project license: MIT, copyright Julian Smolka; retained with downloaded files.
- ROM: `thumb/thumb.gba`, 3,680 bytes.
- SHA-256: `b5cb2291df4ab314b31c598acd9bff2ccfa0b38efff29daadfe97422ce369b67`.
- Selected file hashes and sizes: [`tools/gba-tests-thumb.lock.json`](../tools/gba-tests-thumb.lock.json).
The adapter uses the upstream committed binary. No FASMARM rebuild or source-to-binary reproduction was performed on Darwin.
The entry code initializes the display, switches to Thumb, and initializes r7 to zero.
Each `m_exit` failure writes its test number to r7 and calls `tmain_end`.
Branch tests also use r7 as a progress marker, then reset it to zero on their successful path.
Successful completion of the memory tests reaches the same `tmain_end` with r7 still zero.
At this revision:
- `tmain_end` is Thumb code at `0x08000930`.
- `ADR r0, eval` (`0xa000`) and `BX r0` (`0x4700`) switch execution back to ARM.
- `eval` is ARM code at `0x08000934`, starting with `0xe92d0003`, the `m_vsync` register save.
- The later result macro reads r7, not the ARM suite's r12.
The source exit paths and binary Thumb `BL` targets were checked against `tmain_end`.
The adapter verifies the bridge and initial evaluation word, in addition to whole-file hashes.
The manifest stops before `eval`, requires ARM state and `r7 == 0`, and allows 1,000,000 successful machine steps.
The budget includes the original BIOS boot. Startup and CPU tests execute normally; result rendering does not execute.
Diagnostics, timeouts, STOP, and nonzero r7 remain failures under the [runner's normal rules](rom-tests.md).
## Observed result on Darwin arm64
Initial integration, before fixing empty-list Thumb stores:
- The runner reaches `0x08000934` after 771 successful steps and 3,858 nominal cycles.
- The CPU is in System/ARM state with CPSR `0x0000001f` and r7 = 229.
- Both builds exit nonzero with `reason: assertion_failed`; there is no emulator diagnostic.
- Test 229 executes `STMIA r0!, {}` (`0xc000`) at `0x080008ba`.
- The stored PC value is `0x080008be`, but the following PC read expects `0x080008c0`.
- The public ARM ROM still passes its unchanged checkpoint.
The next CPU task is to verify the empty-list store PC offset and correct it with original regressions.
Do not change the pinned ROM or weaken the result assertion to pass this test.
No public timing, memory, BIOS, graphics, or unsafe suite is claimed as passing.
These results do not establish Pokémon Emerald compatibility.
## References
- [Pinned Thumb entry and result flow](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/thumb.asm).
- [Branch progress markers and successful reset](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/branches.asm).
- [Memory tests, including test 229](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/memory.asm).
- [Shared startup and result macros](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/lib/macros.inc).
- [Pinned license](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/LICENSE).

View file

@ -65,6 +65,7 @@ Hardware references used for the core:
- [GBATEK ARM data processing](https://problemkaputt.de/gbatek-arm-opcodes-data-processing-alu.htm), including R15/status notes. Its legacy `{P}` description alone does not establish ARM7 behavior.
- [mGBA ARM instruction implementation](https://github.com/mgba-emu/mgba/blob/master/src/arm/isa-arm.c), reviewed for Rd=15 test/compare SPSR restoration, User/System fallback, and no result write/refill. Also reviewed its word/byte/halfword/signed load ordering: writeback precedes the destination result.
- [jsmolka/gba-tests](https://github.com/jsmolka/gba-tests/tree/a7113b67e63f83a9b321696ddd7042ccfad6c881), pinned for independent ARM testing. Reviewed its MIT license, startup, result macros, compare/status tests, and load/writeback alias tests.
- [Pinned jsmolka Thumb entry](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/thumb.asm), [branch tests](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/branches.asm), and [memory tests](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/memory.asm), reviewed for r7 failure IDs, success-marker reset, the Thumb-to-ARM evaluation bridge, and empty-list transfer expectations.
- [armwrestler-gba-fixed](https://github.com/destoer/armwrestler-gba-fixed) and [arm7wrestler](https://github.com/Arisotura/arm7wrestler), considered as test sources; not integrated.
- [SingleStepTests ARM7TDMI](https://github.com/SingleStepTests/ARM7TDMI), considered as an experimental, emulator-generated test source; not integrated.
- [GBATEK ARM single data transfers](https://problemkaputt.de/gbatek-arm-opcodes-memory-single-data-transfer-ldr-str-pld.htm), for address calculation, indexing, load extension, and nominal timing.

View file

@ -4,8 +4,9 @@ Use `--test-suite PATH.json` to run repeatable ROM checks without opening a wind
Each case has a machine-step budget, a completion address, and explicit assertions.
A timeout never counts as a pass. Existing `--rom ... --steps ...` remains a diagnostic mode, not a test verdict.
Original programs cover the runner itself. A [pinned public ARM test](public-arm-tests.md) now provides an independent compatibility baseline.
That public ROM still fails; passing the runner's unit tests does not mean the external ARM suite passes.
Original programs cover the runner itself. Pinned public [ARM](public-arm-tests.md) and [Thumb](public-thumb-tests.md) ROMs provide independent compatibility results.
The ARM ROM passes its checkpoint. The Thumb ROM currently reports test 229 as failed.
Passing the runner's unit tests does not mean an external suite passes.
A passing suite establishes only its specified checkpoints and assertions. It does not establish Emerald compatibility.
## Run the original smoke suite
@ -132,7 +133,8 @@ They do not hash ROM contents; record source revisions and file checksums alongs
The first public integration is [jsmolka/gba-tests ARM](public-arm-tests.md), with a verified result-register checkpoint.
The pinned ARM ROM now passes this checkpoint on Darwin arm64. Historical failures remain documented.
Next, integrate a pinned public Thumb suite with a verified result protocol.
The [pinned Thumb ROM](public-thumb-tests.md) also has a verified checkpoint, but currently fails its r7 assertion at test 229.
Verify and correct the empty-list store PC value before claiming that suite passes.
For further public ARM7TDMI/GBA tests, inspect their source, license, entry assumptions, and result protocol.
Pin source revisions and build instructions before comparing emulator changes.
Adapt suites to verified completion addresses and result locations; do not guess them or treat timeouts as passes.

View file

@ -67,7 +67,8 @@ Update this file when a feature lands or a limit is removed.
- ROM windows with scanline capture, keyboard input, focus-loss release, bounded execution slices, and interactive keypad wake from STOP.
- Optional ROM-window frame limits and an original input-test ROM generator.
- Headless JSON ROM suites with per-case checkpoints, register/CPSR/memory assertions, bounded budgets, and structured failure reports.
- Hash-pinned preparation of the public `jsmolka/gba-tests` ARM ROM, passing its result checkpoint on Darwin arm64.
- Hash-pinned preparation of public `jsmolka/gba-tests` ARM and Thumb ROMs, with verified result-register checkpoints.
- The public ARM ROM passes on Darwin arm64; the Thumb ROM has an explicit failing baseline at test 229.
- ARM test/compare Rd=15 status restoration, without a PC result write or nominal refill.
- ARM unused-memory data reads return a PC+8 word snapshot, with normal byte lanes and load rotation.
- Little-endian byte, halfword, and word reads/writes.
@ -115,7 +116,8 @@ ROM windows have no demo-specific startup checks. Input is sampled between bound
Audio, saves, and external BIOS loading remain unavailable. ROM loading does not establish commercial-game compatibility.
See [cartridge loading](hardware/cartridge.md) for file limits, step budgets, and exit status.
The [ROM suite runner](rom-tests.md) has original regressions and a [pinned public ARM baseline](public-arm-tests.md).
The pinned public ARM ROM passes its result checkpoint in debug and release builds. Other public suites remain unverified.
The pinned public ARM ROM passes its result checkpoint in debug and release builds.
The [pinned public Thumb ROM](public-thumb-tests.md) reaches its checkpoint but reports test 229 as failed. Other public suites remain unverified.
The runner requires known completion addresses and has no debug-port protocol, scripted input, or rendered-image assertions.
ARM data reads from unused address ranges use a bounded [PC+8 open-bus snapshot](hardware/cpu.md#arm-unused-memory-data-reads).
@ -128,7 +130,7 @@ Writes to cartridge addresses return errors instead of modeling cartridge hardwa
## Next steps
1. Add a pinned public Thumb suite with a verified result protocol, then address its first compatibility failure.
1. Verify and fix the empty-list Thumb store PC value exposed by public test 229, then rerun both pinned suites.
Extend open-bus support with verified Thumb region/alignment rules and bus history, rather than a blanket unmapped-read fallback.
2. Refine nominal timing with a fetch pipeline, Game Pak prefetch, per-access device updates, and verified timer/IRQ delays.
3. Validate keypad retrigger behavior with hardware tests and add remaining device registers; extend BIOS reset coverage as sound/serial support becomes available.

View file

@ -50,13 +50,15 @@ The [headless ROM suite runner](rom-tests.md) provides explicit checkpoint and a
Its tests verify manifest limits, whole-suite validation, exact step boundaries, ARM/Thumb checkpoints, and register/CPSR/memory checks.
Additional tests cover DMA/IRQ accounting, HALT/STOP, diagnostic state, case isolation, repeatable JSON output, and failure exit status.
These original tests validate the runner. The [public ARM adapter](public-arm-tests.md) provides a separate passing checkpoint result for its pinned ROM.
Its preparation tests use only synthetic original bytes and no network:
The [public Thumb adapter](public-thumb-tests.md) uses a separate lock and r7 assertion after its return to ARM state.
Preparation tests use only synthetic original bytes and no network:
```sh
direnv exec . python3 -B -m unittest discover -s tools -p 'test_*.py' -v
```
They verify pinned URLs, hashes, sizes, checkpoint validation, offline copies, destination safety, and non-overwriting output.
They verify suite selection, pinned URLs, hashes, sizes, checkpoint validation, offline copies, destination safety, and non-overwriting output.
Thumb cases also verify the return-to-ARM bridge. Command-line tests cover default ARM selection, explicit Thumb selection, and errors.
Compare/status CPU regressions verify saved flags/masks, bank switching, sequential PC/timing, User/System fallback, and atomic diagnostics.
Load-alias regressions cover widths, signed/unaligned values, immediate/register offsets, pre/post-indexing, every non-PC register bank, and address wrapping.
They verify loaded-value precedence, preserved flags, unchanged stores, nominal timing, I/O reads before device progress, deferred IRQ delivery, and atomic errors.