test(cpu): integrate pinned public Thumb compatibility suite
Add --suite thumb while retaining default ARM preparation. Verify hashes, the Thumb-to-ARM evaluation bridge, and the result checkpoint; retain upstream provenance outside tracked files. Expand offline synthetic adapter and CLI regressions. Darwin arm64 baseline: the unmodified Thumb ROM reaches ARM eval at 0x08000934 after 771 steps with r7=229 in debug and release. ARM remains passing. Online/offline preparation agrees and all 21 Python tests pass. CPU behavior is unchanged.
This commit is contained in:
parent
a9493d4cc6
commit
08c8705904
11 changed files with 304 additions and 50 deletions
|
|
@ -14,7 +14,7 @@ The repository is a Cargo workspace with three crates:
|
|||
| Path | Purpose |
|
||||
| --- | --- |
|
||||
| `flake.nix`, `flake.lock` | Pinned Rust/Python development environment |
|
||||
| `tools/prepare_gba_tests.py`, `tools/gba-tests-arm.lock.json` | Hash-verified public ARM source/ROM preparation and suite generation |
|
||||
| `tools/prepare_gba_tests.py`, `tools/gba-tests-{arm,thumb}.lock.json` | Hash-verified public ARM/Thumb source/ROM preparation and suite generation |
|
||||
| `tools/test_prepare_gba_tests.py` | Offline adapter tests using original synthetic data |
|
||||
| `.envrc` | Automatic environment loading through direnv |
|
||||
| `AGENTS.md` | Short orientation for coding agents: crate map, commands, and where docs live |
|
||||
|
|
|
|||
81
docs/public-thumb-tests.md
Normal file
81
docs/public-thumb-tests.md
Normal file
|
|
@ -0,0 +1,81 @@
|
|||
# Pinned public Thumb test
|
||||
|
||||
The headless runner executes the unmodified Thumb ROM from [jsmolka/gba-tests](https://github.com/jsmolka/gba-tests).
|
||||
It currently reaches its result checkpoint but fails test 229 on Darwin arm64 in debug and release builds.
|
||||
This is an independent compatibility baseline, not complete ARM7TDMI conformance or a hardware measurement.
|
||||
|
||||
## Prepare and run
|
||||
|
||||
```sh
|
||||
direnv exec . python3 tools/prepare_gba_tests.py /tmp/gba-public-thumb --suite thumb
|
||||
direnv exec . cargo run --locked --release -- --test-suite /tmp/gba-public-thumb/suite.json > /tmp/gba-public-thumb-report.json
|
||||
```
|
||||
|
||||
Use a new directory outside the repository or inside ignored `roms/`. The parent directory must exist.
|
||||
The adapter checks every selected file's byte count and SHA-256 digest before creating output.
|
||||
It retains the upstream sources, license, README, ROM, generated `suite.json`, and `source-lock.json`.
|
||||
No source, binary, header, expected result, or instruction is patched or skipped.
|
||||
Downloaded binaries and header sources must not enter tracked files.
|
||||
|
||||
Offline preparation uses the same checks:
|
||||
|
||||
```sh
|
||||
direnv exec . python3 tools/prepare_gba_tests.py /tmp/gba-public-thumb-offline --suite thumb --source-dir /path/to/gba-tests
|
||||
```
|
||||
|
||||
The existing [ARM command](public-arm-tests.md) remains the default. Explicit `--suite arm` selects the same ARM lock.
|
||||
Each preparation creates one suite. The suite name does not select the checkpoint's instruction state.
|
||||
Python comes from Nix and requires no extra packages. Ordinary unit tests use original synthetic data, without downloads.
|
||||
Preparation is not a compatibility verdict. Inspect the runner's JSON result and exit status.
|
||||
|
||||
## Provenance and result protocol
|
||||
|
||||
- Repository: `jsmolka/gba-tests`.
|
||||
- Revision: `a7113b67e63f83a9b321696ddd7042ccfad6c881`.
|
||||
- Upstream project license: MIT, copyright Julian Smolka; retained with downloaded files.
|
||||
- ROM: `thumb/thumb.gba`, 3,680 bytes.
|
||||
- SHA-256: `b5cb2291df4ab314b31c598acd9bff2ccfa0b38efff29daadfe97422ce369b67`.
|
||||
- Selected file hashes and sizes: [`tools/gba-tests-thumb.lock.json`](../tools/gba-tests-thumb.lock.json).
|
||||
|
||||
The adapter uses the upstream committed binary. No FASMARM rebuild or source-to-binary reproduction was performed on Darwin.
|
||||
The entry code initializes the display, switches to Thumb, and initializes r7 to zero.
|
||||
Each `m_exit` failure writes its test number to r7 and calls `tmain_end`.
|
||||
Branch tests also use r7 as a progress marker, then reset it to zero on their successful path.
|
||||
Successful completion of the memory tests reaches the same `tmain_end` with r7 still zero.
|
||||
|
||||
At this revision:
|
||||
|
||||
- `tmain_end` is Thumb code at `0x08000930`.
|
||||
- `ADR r0, eval` (`0xa000`) and `BX r0` (`0x4700`) switch execution back to ARM.
|
||||
- `eval` is ARM code at `0x08000934`, starting with `0xe92d0003`, the `m_vsync` register save.
|
||||
- The later result macro reads r7, not the ARM suite's r12.
|
||||
|
||||
The source exit paths and binary Thumb `BL` targets were checked against `tmain_end`.
|
||||
The adapter verifies the bridge and initial evaluation word, in addition to whole-file hashes.
|
||||
The manifest stops before `eval`, requires ARM state and `r7 == 0`, and allows 1,000,000 successful machine steps.
|
||||
The budget includes the original BIOS boot. Startup and CPU tests execute normally; result rendering does not execute.
|
||||
Diagnostics, timeouts, STOP, and nonzero r7 remain failures under the [runner's normal rules](rom-tests.md).
|
||||
|
||||
## Observed result on Darwin arm64
|
||||
|
||||
Initial integration, before fixing empty-list Thumb stores:
|
||||
|
||||
- The runner reaches `0x08000934` after 771 successful steps and 3,858 nominal cycles.
|
||||
- The CPU is in System/ARM state with CPSR `0x0000001f` and r7 = 229.
|
||||
- Both builds exit nonzero with `reason: assertion_failed`; there is no emulator diagnostic.
|
||||
- Test 229 executes `STMIA r0!, {}` (`0xc000`) at `0x080008ba`.
|
||||
- The stored PC value is `0x080008be`, but the following PC read expects `0x080008c0`.
|
||||
- The public ARM ROM still passes its unchanged checkpoint.
|
||||
|
||||
The next CPU task is to verify the empty-list store PC offset and correct it with original regressions.
|
||||
Do not change the pinned ROM or weaken the result assertion to pass this test.
|
||||
No public timing, memory, BIOS, graphics, or unsafe suite is claimed as passing.
|
||||
These results do not establish Pokémon Emerald compatibility.
|
||||
|
||||
## References
|
||||
|
||||
- [Pinned Thumb entry and result flow](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/thumb.asm).
|
||||
- [Branch progress markers and successful reset](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/branches.asm).
|
||||
- [Memory tests, including test 229](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/memory.asm).
|
||||
- [Shared startup and result macros](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/lib/macros.inc).
|
||||
- [Pinned license](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/LICENSE).
|
||||
|
|
@ -65,6 +65,7 @@ Hardware references used for the core:
|
|||
- [GBATEK ARM data processing](https://problemkaputt.de/gbatek-arm-opcodes-data-processing-alu.htm), including R15/status notes. Its legacy `{P}` description alone does not establish ARM7 behavior.
|
||||
- [mGBA ARM instruction implementation](https://github.com/mgba-emu/mgba/blob/master/src/arm/isa-arm.c), reviewed for Rd=15 test/compare SPSR restoration, User/System fallback, and no result write/refill. Also reviewed its word/byte/halfword/signed load ordering: writeback precedes the destination result.
|
||||
- [jsmolka/gba-tests](https://github.com/jsmolka/gba-tests/tree/a7113b67e63f83a9b321696ddd7042ccfad6c881), pinned for independent ARM testing. Reviewed its MIT license, startup, result macros, compare/status tests, and load/writeback alias tests.
|
||||
- [Pinned jsmolka Thumb entry](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/thumb.asm), [branch tests](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/branches.asm), and [memory tests](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/memory.asm), reviewed for r7 failure IDs, success-marker reset, the Thumb-to-ARM evaluation bridge, and empty-list transfer expectations.
|
||||
- [armwrestler-gba-fixed](https://github.com/destoer/armwrestler-gba-fixed) and [arm7wrestler](https://github.com/Arisotura/arm7wrestler), considered as test sources; not integrated.
|
||||
- [SingleStepTests ARM7TDMI](https://github.com/SingleStepTests/ARM7TDMI), considered as an experimental, emulator-generated test source; not integrated.
|
||||
- [GBATEK ARM single data transfers](https://problemkaputt.de/gbatek-arm-opcodes-memory-single-data-transfer-ldr-str-pld.htm), for address calculation, indexing, load extension, and nominal timing.
|
||||
|
|
|
|||
|
|
@ -4,8 +4,9 @@ Use `--test-suite PATH.json` to run repeatable ROM checks without opening a wind
|
|||
Each case has a machine-step budget, a completion address, and explicit assertions.
|
||||
A timeout never counts as a pass. Existing `--rom ... --steps ...` remains a diagnostic mode, not a test verdict.
|
||||
|
||||
Original programs cover the runner itself. A [pinned public ARM test](public-arm-tests.md) now provides an independent compatibility baseline.
|
||||
That public ROM still fails; passing the runner's unit tests does not mean the external ARM suite passes.
|
||||
Original programs cover the runner itself. Pinned public [ARM](public-arm-tests.md) and [Thumb](public-thumb-tests.md) ROMs provide independent compatibility results.
|
||||
The ARM ROM passes its checkpoint. The Thumb ROM currently reports test 229 as failed.
|
||||
Passing the runner's unit tests does not mean an external suite passes.
|
||||
A passing suite establishes only its specified checkpoints and assertions. It does not establish Emerald compatibility.
|
||||
|
||||
## Run the original smoke suite
|
||||
|
|
@ -132,7 +133,8 @@ They do not hash ROM contents; record source revisions and file checksums alongs
|
|||
|
||||
The first public integration is [jsmolka/gba-tests ARM](public-arm-tests.md), with a verified result-register checkpoint.
|
||||
The pinned ARM ROM now passes this checkpoint on Darwin arm64. Historical failures remain documented.
|
||||
Next, integrate a pinned public Thumb suite with a verified result protocol.
|
||||
The [pinned Thumb ROM](public-thumb-tests.md) also has a verified checkpoint, but currently fails its r7 assertion at test 229.
|
||||
Verify and correct the empty-list store PC value before claiming that suite passes.
|
||||
For further public ARM7TDMI/GBA tests, inspect their source, license, entry assumptions, and result protocol.
|
||||
Pin source revisions and build instructions before comparing emulator changes.
|
||||
Adapt suites to verified completion addresses and result locations; do not guess them or treat timeouts as passes.
|
||||
|
|
|
|||
|
|
@ -67,7 +67,8 @@ Update this file when a feature lands or a limit is removed.
|
|||
- ROM windows with scanline capture, keyboard input, focus-loss release, bounded execution slices, and interactive keypad wake from STOP.
|
||||
- Optional ROM-window frame limits and an original input-test ROM generator.
|
||||
- Headless JSON ROM suites with per-case checkpoints, register/CPSR/memory assertions, bounded budgets, and structured failure reports.
|
||||
- Hash-pinned preparation of the public `jsmolka/gba-tests` ARM ROM, passing its result checkpoint on Darwin arm64.
|
||||
- Hash-pinned preparation of public `jsmolka/gba-tests` ARM and Thumb ROMs, with verified result-register checkpoints.
|
||||
- The public ARM ROM passes on Darwin arm64; the Thumb ROM has an explicit failing baseline at test 229.
|
||||
- ARM test/compare Rd=15 status restoration, without a PC result write or nominal refill.
|
||||
- ARM unused-memory data reads return a PC+8 word snapshot, with normal byte lanes and load rotation.
|
||||
- Little-endian byte, halfword, and word reads/writes.
|
||||
|
|
@ -115,7 +116,8 @@ ROM windows have no demo-specific startup checks. Input is sampled between bound
|
|||
Audio, saves, and external BIOS loading remain unavailable. ROM loading does not establish commercial-game compatibility.
|
||||
See [cartridge loading](hardware/cartridge.md) for file limits, step budgets, and exit status.
|
||||
The [ROM suite runner](rom-tests.md) has original regressions and a [pinned public ARM baseline](public-arm-tests.md).
|
||||
The pinned public ARM ROM passes its result checkpoint in debug and release builds. Other public suites remain unverified.
|
||||
The pinned public ARM ROM passes its result checkpoint in debug and release builds.
|
||||
The [pinned public Thumb ROM](public-thumb-tests.md) reaches its checkpoint but reports test 229 as failed. Other public suites remain unverified.
|
||||
The runner requires known completion addresses and has no debug-port protocol, scripted input, or rendered-image assertions.
|
||||
|
||||
ARM data reads from unused address ranges use a bounded [PC+8 open-bus snapshot](hardware/cpu.md#arm-unused-memory-data-reads).
|
||||
|
|
@ -128,7 +130,7 @@ Writes to cartridge addresses return errors instead of modeling cartridge hardwa
|
|||
|
||||
## Next steps
|
||||
|
||||
1. Add a pinned public Thumb suite with a verified result protocol, then address its first compatibility failure.
|
||||
1. Verify and fix the empty-list Thumb store PC value exposed by public test 229, then rerun both pinned suites.
|
||||
Extend open-bus support with verified Thumb region/alignment rules and bus history, rather than a blanket unmapped-read fallback.
|
||||
2. Refine nominal timing with a fetch pipeline, Game Pak prefetch, per-access device updates, and verified timer/IRQ delays.
|
||||
3. Validate keypad retrigger behavior with hardware tests and add remaining device registers; extend BIOS reset coverage as sound/serial support becomes available.
|
||||
|
|
|
|||
|
|
@ -50,13 +50,15 @@ The [headless ROM suite runner](rom-tests.md) provides explicit checkpoint and a
|
|||
Its tests verify manifest limits, whole-suite validation, exact step boundaries, ARM/Thumb checkpoints, and register/CPSR/memory checks.
|
||||
Additional tests cover DMA/IRQ accounting, HALT/STOP, diagnostic state, case isolation, repeatable JSON output, and failure exit status.
|
||||
These original tests validate the runner. The [public ARM adapter](public-arm-tests.md) provides a separate passing checkpoint result for its pinned ROM.
|
||||
Its preparation tests use only synthetic original bytes and no network:
|
||||
The [public Thumb adapter](public-thumb-tests.md) uses a separate lock and r7 assertion after its return to ARM state.
|
||||
Preparation tests use only synthetic original bytes and no network:
|
||||
|
||||
```sh
|
||||
direnv exec . python3 -B -m unittest discover -s tools -p 'test_*.py' -v
|
||||
```
|
||||
|
||||
They verify pinned URLs, hashes, sizes, checkpoint validation, offline copies, destination safety, and non-overwriting output.
|
||||
They verify suite selection, pinned URLs, hashes, sizes, checkpoint validation, offline copies, destination safety, and non-overwriting output.
|
||||
Thumb cases also verify the return-to-ARM bridge. Command-line tests cover default ARM selection, explicit Thumb selection, and errors.
|
||||
Compare/status CPU regressions verify saved flags/masks, bank switching, sequential PC/timing, User/System fallback, and atomic diagnostics.
|
||||
Load-alias regressions cover widths, signed/unaligned values, immediate/register offsets, pre/post-indexing, every non-PC register bank, and address wrapping.
|
||||
They verify loaded-value precedence, preserved flags, unchanged stores, nominal timing, I/O reads before device progress, deferred IRQ delivery, and atomic errors.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue