test(memory): integrate pinned public memory compatibility suite
Add --suite memory with verified source hashes, ARM eval checkpoint 0x08000350, and the r12 result assertion. Keep default ARM and explicit Thumb preparation unchanged. Extend offline adapter and CLI tests; retain public binaries outside tracked files. The unmodified memory ROM passes in debug and release after 174 steps and 1478 nominal cycles, without changing core behavior. Document weak byte-write assertions and mode-selection caveats. Add original CPU-driven tests with exact sentinels, mode/mirror boundaries, byte lanes, forced blank, readback, and nominal device timing. Validated on Darwin arm64: 851 workspace tests in debug and release; 782 core/demo tests separately; 30 Python tests; all three pinned public suites in both profiles; native ROM windows; all ten demo smoke tests; fmt, clippy, rustdoc, and local documentation links.
This commit is contained in:
parent
3d67b136ba
commit
661f00d682
14 changed files with 351 additions and 19 deletions
|
|
@ -14,7 +14,7 @@ The repository is a Cargo workspace with three crates:
|
|||
| Path | Purpose |
|
||||
| --- | --- |
|
||||
| `flake.nix`, `flake.lock` | Pinned Rust/Python development environment |
|
||||
| `tools/prepare_gba_tests.py`, `tools/gba-tests-{arm,thumb}.lock.json` | Hash-verified public ARM/Thumb source/ROM preparation and suite generation |
|
||||
| `tools/prepare_gba_tests.py`, `tools/gba-tests-{arm,thumb,memory}.lock.json` | Hash-verified public test source/ROM preparation and suite generation |
|
||||
| `tools/test_prepare_gba_tests.py` | Offline adapter tests using original synthetic data |
|
||||
| `.envrc` | Automatic environment loading through direnv |
|
||||
| `AGENTS.md` | Short orientation for coding agents: crate map, commands, and where docs live |
|
||||
|
|
@ -98,6 +98,7 @@ The repository is a Cargo workspace with three crates:
|
|||
| `crates/desktop/tests/rom_cli.rs` | File loading and exit-status checks; opt-in native ROM-window integration test |
|
||||
| `crates/gba-core/tests/core.rs` | CPU integration tests and the complete demo |
|
||||
| `crates/gba-core/tests/memory.rs` | Memory widths, alignment, errors, and mirrors |
|
||||
| `crates/gba-core/tests/video_bus_cpu.rs` | Original CPU video-byte writes, exact sentinels, mirror/mode boundaries, readback, and nominal store timing |
|
||||
| `crates/gba-core/tests/exceptions.rs` | Exception demo, BIOS mapping, and reset-vector execution |
|
||||
| `crates/gba-core/tests/timers.rs` | Timer rules, interrupt registers, and a cycle-by-cycle reference |
|
||||
| `crates/gba-core/tests/machine.rs` | Device IRQ entry/return, clock policy, and I/O failure atomicity |
|
||||
|
|
|
|||
|
|
@ -111,7 +111,8 @@ Original regressions separately check the returned word, byte lanes, rotation, s
|
|||
See the bounded [ARM open-bus implementation](hardware/cpu.md#arm-unused-memory-data-reads).
|
||||
This is a passing result for this pinned ARM ROM, not proof of complete instruction, memory, or timing compatibility.
|
||||
The [pinned public Thumb ROM](public-thumb-tests.md) also passes its verified checkpoint.
|
||||
No public timing, memory, BIOS, graphics, or unsafe suite has been claimed as passing.
|
||||
The [pinned public memory ROM](public-memory-tests.md) passes its limited mirror and video-byte checks.
|
||||
No public timing, BIOS, graphics, save, or unsafe suite has been claimed as passing.
|
||||
These results do not establish Pokémon Emerald compatibility.
|
||||
|
||||
## References
|
||||
|
|
|
|||
94
docs/public-memory-tests.md
Normal file
94
docs/public-memory-tests.md
Normal file
|
|
@ -0,0 +1,94 @@
|
|||
# Pinned public memory test
|
||||
|
||||
The headless runner executes the unmodified memory ROM from [jsmolka/gba-tests](https://github.com/jsmolka/gba-tests).
|
||||
It passes its result checkpoint on Darwin arm64 in debug and release builds without a core behavior change.
|
||||
This suite samples memory mirrors and video-byte writes. It does not establish complete memory-bus compatibility.
|
||||
|
||||
## Prepare and run
|
||||
|
||||
```sh
|
||||
direnv exec . python3 tools/prepare_gba_tests.py /tmp/gba-public-memory --suite memory
|
||||
direnv exec . cargo run --locked --release -- --test-suite /tmp/gba-public-memory/suite.json > /tmp/gba-public-memory-report.json
|
||||
```
|
||||
|
||||
Use a new directory outside the repository or inside ignored `roms/`. Its parent directory must exist.
|
||||
Every selected source and binary file has a pinned byte count and SHA-256 digest.
|
||||
The adapter checks all files before creating output, retains the upstream license, and writes `suite.json` and `source-lock.json`.
|
||||
No ROM patch, source rewrite, header replacement, test skip, or result adjustment occurs.
|
||||
Downloaded ROMs and header sources must not enter tracked files.
|
||||
|
||||
Offline preparation applies the same checks:
|
||||
|
||||
```sh
|
||||
direnv exec . python3 tools/prepare_gba_tests.py /tmp/gba-public-memory-offline --suite memory --source-dir /path/to/gba-tests
|
||||
```
|
||||
|
||||
The default remains the [ARM suite](public-arm-tests.md). `--suite thumb` selects the [Thumb suite](public-thumb-tests.md).
|
||||
Each invocation prepares one suite. Normal Rust and Python tests do not download public files.
|
||||
Python comes from Nix and requires no extra packages. Preparation alone is not a compatibility verdict.
|
||||
|
||||
## Provenance and result protocol
|
||||
|
||||
- Repository: `jsmolka/gba-tests`.
|
||||
- Revision: `a7113b67e63f83a9b321696ddd7042ccfad6c881`.
|
||||
- Upstream project license: MIT, copyright Julian Smolka; retained with downloaded files.
|
||||
- ROM: `memory/memory.gba`, 2,172 bytes.
|
||||
- SHA-256: `21024fb6aae6343f5f0466dd54e3149de1fbeb23f78e7d85a015c983684d2f87`.
|
||||
- Selected file hashes and sizes: [`tools/gba-tests-memory.lock.json`](../tools/gba-tests-memory.lock.json).
|
||||
|
||||
The adapter uses the committed upstream binary. No FASMARM rebuild or source-to-binary reproduction was performed on Darwin.
|
||||
Startup initializes Mode 4 display state and sets r12 to zero.
|
||||
Failure paths set r12 to their test number and branch to `eval`.
|
||||
Successful execution reaches `eval` with r12 still zero.
|
||||
The source exits and their binary ARM branch targets were checked against this address.
|
||||
|
||||
`eval` is `0x08000350`, in ARM state. Its first word is `0xe92d0003`, the shared `m_vsync` register save.
|
||||
Preparation checks this word in addition to whole-file hashes.
|
||||
The manifest stops before executing `eval` and requires `r12 == 0`.
|
||||
This runs startup and all memory tests without depending on result rendering, VBlank polling, or BIOS number formatting.
|
||||
The budget is 1,000,000 successful machine steps, including original BIOS boot.
|
||||
Diagnostics, timeouts, STOP, and failed assertions remain failures under the [runner's normal rules](rom-tests.md).
|
||||
|
||||
## Observed result on Darwin arm64
|
||||
|
||||
- The ROM reaches `0x08000350` after 174 successful steps and 1,478 nominal cycles.
|
||||
- The CPU is in System/ARM state, with CPSR `0x6000001f` and r12 = 0.
|
||||
- Debug and release builds exit zero with `reason: checkpoint`, no error, and the result assertion passing.
|
||||
- The existing ARM and Thumb suites also retain their passing results.
|
||||
- No emulator behavior change was needed for this result.
|
||||
|
||||
## Coverage and limits
|
||||
|
||||
The mirror tests sample external RAM, internal RAM, palette RAM, video RAM, object attribute memory, and both alternate cartridge windows.
|
||||
The byte-write tests sample palette duplication, background video RAM duplication, and ignored object-memory writes.
|
||||
They do not cover open bus, BIOS protection, save hardware, unused I/O, memory timing, or display-bus contention.
|
||||
|
||||
The pinned source also has limits that weaken its video-byte checks:
|
||||
|
||||
- Tests 50–52 reject one exact byte-sized value, rather than requiring the complete prior word to remain unchanged.
|
||||
An incorrect duplicated-halfword write can therefore escape these checks.
|
||||
- Startup selects Mode 4. Test 51 applies `ORR #3` to that mode, selecting reserved Mode 7 rather than a supported bitmap mode.
|
||||
- Test 52 clears only mode bits 0–1. This leaves Mode 4 selected rather than selecting a tile mode.
|
||||
- Test 5 explicitly selects Mode 0 for its upper video RAM mirror check. It does not validate every mirror in every mode.
|
||||
|
||||
These source behaviors are retained, not patched to make the suite stronger or easier to pass.
|
||||
Passing this ROM must not be described as exhaustive video-memory validation.
|
||||
|
||||
Original CPU-driven regressions in `crates/gba-core/tests/video_bus_cpu.rs` provide separate, stronger byte-write checks.
|
||||
They select each supported mode explicitly and require the exact duplicated halfword or unchanged nonzero sentinel.
|
||||
They cover both byte lanes, physical and mirrored addresses, tile/bitmap boundaries, forced blank, and CPU readback.
|
||||
Adjacent halfwords must remain unchanged. Ignored stores still pay their nominal data-access cost and advance devices.
|
||||
Halfword and word stores still modify object video RAM and object attribute memory.
|
||||
These are implementation regressions based on documented rules, not new hardware measurements.
|
||||
|
||||
No public BIOS, timing, graphics, save, or unsafe suite is claimed as passing.
|
||||
Pokémon Emerald compatibility remains unverified.
|
||||
|
||||
## References
|
||||
|
||||
- [Pinned memory entry and result flow](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/memory/memory.asm).
|
||||
- [Memory mirror checks](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/memory/mirrors.asm).
|
||||
- [Video-byte checks and mode selection](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/memory/video_strb.asm).
|
||||
- [Display initialization](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/lib/text.asm) and [shared result macros](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/lib/macros.inc).
|
||||
- [GBATEK memory mirrors and video-byte writes](https://problemkaputt.de/gbatek-gba-unpredictable-things.htm).
|
||||
- [Pinned license](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/LICENSE).
|
||||
|
|
@ -81,7 +81,8 @@ They cover all low base registers, banked stack pointers, code/data alignment, R
|
|||
Existing empty-list expectations that incorrectly required PC+4 were corrected; empty loads and ARM stores retain their behavior.
|
||||
See [CPU transfer semantics](hardware/cpu.md#instruction-demo-and-execution-rules).
|
||||
This proves only the pinned suites' checkpoints and assertions, not complete CPU, bus, or timing compatibility.
|
||||
No public timing, memory, BIOS, graphics, or unsafe suite is claimed as passing.
|
||||
The [pinned public memory ROM](public-memory-tests.md) also passes, with documented video-byte coverage limits.
|
||||
No public timing, BIOS, graphics, save, or unsafe suite is claimed as passing.
|
||||
These results do not establish Pokémon Emerald compatibility.
|
||||
|
||||
## References
|
||||
|
|
|
|||
|
|
@ -66,6 +66,8 @@ Hardware references used for the core:
|
|||
- [mGBA ARM instruction implementation](https://github.com/mgba-emu/mgba/blob/master/src/arm/isa-arm.c), reviewed for Rd=15 test/compare SPSR restoration, User/System fallback, and no result write/refill. Also reviewed its word/byte/halfword/signed load ordering: writeback precedes the destination result.
|
||||
- [jsmolka/gba-tests](https://github.com/jsmolka/gba-tests/tree/a7113b67e63f83a9b321696ddd7042ccfad6c881), pinned for independent ARM testing. Reviewed its MIT license, startup, result macros, compare/status tests, and load/writeback alias tests.
|
||||
- [Pinned jsmolka Thumb entry](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/thumb.asm), [branch tests](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/branches.asm), and [memory tests](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/memory.asm), reviewed for r7 failure IDs, success-marker reset, the Thumb-to-ARM evaluation bridge, and empty-list transfer expectations.
|
||||
- [Pinned jsmolka memory entry](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/memory/memory.asm), [mirror tests](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/memory/mirrors.asm), and [video-byte tests](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/memory/video_strb.asm), reviewed for the r12 checkpoint, binary exit targets, limited byte-write assertions, and mode-selection caveats.
|
||||
- [gbadoc memory layout](https://gbadev.net/gbadoc/memory.html), consulted for VRAM mirror ranges alongside GBATEK's video-memory layout and byte-write rules.
|
||||
- [armwrestler-gba-fixed](https://github.com/destoer/armwrestler-gba-fixed) and [arm7wrestler](https://github.com/Arisotura/arm7wrestler), considered as test sources; not integrated.
|
||||
- [SingleStepTests ARM7TDMI](https://github.com/SingleStepTests/ARM7TDMI), considered as an experimental, emulator-generated test source; not integrated.
|
||||
- [GBATEK ARM single data transfers](https://problemkaputt.de/gbatek-arm-opcodes-memory-single-data-transfer-ldr-str-pld.htm), for address calculation, indexing, load extension, and nominal timing.
|
||||
|
|
|
|||
|
|
@ -4,8 +4,9 @@ Use `--test-suite PATH.json` to run repeatable ROM checks without opening a wind
|
|||
Each case has a machine-step budget, a completion address, and explicit assertions.
|
||||
A timeout never counts as a pass. Existing `--rom ... --steps ...` remains a diagnostic mode, not a test verdict.
|
||||
|
||||
Original programs cover the runner itself. Pinned public [ARM](public-arm-tests.md) and [Thumb](public-thumb-tests.md) ROMs provide independent compatibility results.
|
||||
Both pinned ROMs pass their checkpoints on Darwin arm64 in debug and release builds.
|
||||
Original programs cover the runner itself. Pinned public [ARM](public-arm-tests.md), [Thumb](public-thumb-tests.md), and [memory](public-memory-tests.md) ROMs provide independent compatibility results.
|
||||
These pinned ROMs pass their checkpoints on Darwin arm64 in debug and release builds.
|
||||
The memory ROM has documented assertion and mode-selection limits; a pass is not exhaustive memory validation.
|
||||
Passing the runner's unit tests does not mean an external suite passes.
|
||||
A passing suite establishes only its specified checkpoints and assertions. It does not establish Emerald compatibility.
|
||||
|
||||
|
|
@ -134,7 +135,8 @@ They do not hash ROM contents; record source revisions and file checksums alongs
|
|||
The first public integration is [jsmolka/gba-tests ARM](public-arm-tests.md), with a verified result-register checkpoint.
|
||||
The pinned ARM ROM now passes this checkpoint on Darwin arm64. Historical failures remain documented.
|
||||
The [pinned Thumb ROM](public-thumb-tests.md) also passes its verified r7 checkpoint after the empty-list store correction.
|
||||
Next, integrate a pinned public memory suite with a verified result protocol.
|
||||
The [pinned memory ROM](public-memory-tests.md) also passes, without a core change and with explicit coverage limits.
|
||||
Next, integrate a pinned public BIOS suite with a verified result protocol.
|
||||
For further public ARM7TDMI/GBA tests, inspect their source, license, entry assumptions, and result protocol.
|
||||
Pin source revisions and build instructions before comparing emulator changes.
|
||||
Adapt suites to verified completion addresses and result locations; do not guess them or treat timeouts as passes.
|
||||
|
|
|
|||
|
|
@ -67,8 +67,8 @@ Update this file when a feature lands or a limit is removed.
|
|||
- ROM windows with scanline capture, keyboard input, focus-loss release, bounded execution slices, and interactive keypad wake from STOP.
|
||||
- Optional ROM-window frame limits and an original input-test ROM generator.
|
||||
- Headless JSON ROM suites with per-case checkpoints, register/CPSR/memory assertions, bounded budgets, and structured failure reports.
|
||||
- Hash-pinned preparation of public `jsmolka/gba-tests` ARM and Thumb ROMs, with verified result-register checkpoints.
|
||||
- The pinned public ARM and Thumb ROMs pass their checkpoints on Darwin arm64 in debug and release builds.
|
||||
- Hash-pinned preparation of public `jsmolka/gba-tests` ARM, Thumb, and memory ROMs, with verified result-register checkpoints.
|
||||
- These pinned ROMs pass their checkpoints on Darwin arm64 in debug and release builds; coverage limits remain documented.
|
||||
- ARM test/compare Rd=15 status restoration, without a PC result write or nominal refill.
|
||||
- ARM unused-memory data reads return a PC+8 word snapshot, with normal byte lanes and load rotation.
|
||||
- Little-endian byte, halfword, and word reads/writes.
|
||||
|
|
@ -118,7 +118,8 @@ See [cartridge loading](hardware/cartridge.md) for file limits, step budgets, an
|
|||
The [ROM suite runner](rom-tests.md) has original regressions and a [pinned public ARM baseline](public-arm-tests.md).
|
||||
The pinned public ARM ROM passes its result checkpoint in debug and release builds.
|
||||
The [pinned public Thumb ROM](public-thumb-tests.md) also passes after correcting its empty-list store PC value.
|
||||
Other public suites remain unverified. These results do not establish full CPU compatibility.
|
||||
The [pinned public memory ROM](public-memory-tests.md) passes without a core change; its video-byte assertions have documented limits.
|
||||
Other public suites remain unverified. These results do not establish full CPU or memory compatibility.
|
||||
The runner requires known completion addresses and has no debug-port protocol, scripted input, or rendered-image assertions.
|
||||
|
||||
ARM data reads from unused address ranges use a bounded [PC+8 open-bus snapshot](hardware/cpu.md#arm-unused-memory-data-reads).
|
||||
|
|
@ -131,7 +132,7 @@ Writes to cartridge addresses return errors instead of modeling cartridge hardwa
|
|||
|
||||
## Next steps
|
||||
|
||||
1. Add a pinned public memory suite with a verified result protocol, then address its first compatibility failure.
|
||||
1. Add a pinned public BIOS suite with a verified result protocol, then address its first compatibility failure.
|
||||
Extend open-bus support with verified Thumb region/alignment rules and bus history, rather than a blanket unmapped-read fallback.
|
||||
2. Refine nominal timing with a fetch pipeline, Game Pak prefetch, per-access device updates, and verified timer/IRQ delays.
|
||||
3. Validate keypad retrigger behavior with hardware tests and add remaining device registers; extend BIOS reset coverage as sound/serial support becomes available.
|
||||
|
|
|
|||
|
|
@ -51,6 +51,7 @@ Its tests verify manifest limits, whole-suite validation, exact step boundaries,
|
|||
Additional tests cover DMA/IRQ accounting, HALT/STOP, diagnostic state, case isolation, repeatable JSON output, and failure exit status.
|
||||
These original tests validate the runner. The [public ARM adapter](public-arm-tests.md) provides a separate passing checkpoint result for its pinned ROM.
|
||||
The [public Thumb adapter](public-thumb-tests.md) uses a separate lock and r7 assertion after its return to ARM state.
|
||||
The [public memory adapter](public-memory-tests.md) uses its own lock and an ARM-state r12 checkpoint.
|
||||
Preparation tests use only synthetic original bytes and no network:
|
||||
|
||||
```sh
|
||||
|
|
@ -58,7 +59,7 @@ direnv exec . python3 -B -m unittest discover -s tools -p 'test_*.py' -v
|
|||
```
|
||||
|
||||
They verify suite selection, pinned URLs, hashes, sizes, checkpoint validation, offline copies, destination safety, and non-overwriting output.
|
||||
Thumb cases also verify the return-to-ARM bridge. Command-line tests cover default ARM selection, explicit Thumb selection, and errors.
|
||||
Thumb cases also verify the return-to-ARM bridge. Command-line tests cover default ARM selection, explicit Thumb/memory selection, and errors.
|
||||
Compare/status CPU regressions verify saved flags/masks, bank switching, sequential PC/timing, User/System fallback, and atomic diagnostics.
|
||||
Load-alias regressions cover widths, signed/unaligned values, immediate/register offsets, pre/post-indexing, every non-PC register bank, and address wrapping.
|
||||
They verify loaded-value precedence, preserved flags, unchanged stores, nominal timing, I/O reads before device progress, deferred IRQ delivery, and atomic errors.
|
||||
|
|
@ -69,7 +70,12 @@ The pinned public Thumb ROM also reaches its ARM-state checkpoint with r7 = 0 in
|
|||
Empty-list Thumb regressions cover PC+6, all low bases and processor modes, banked stacks, both code alignments, and unaligned data.
|
||||
They check one-word writes, 64-byte writeback, ROM windows, the following PC operand, nominal timing, and atomic errors.
|
||||
Existing ARM empty-list and Thumb empty-load tests continue to pass. `PUSH {lr}` retains its normal link value.
|
||||
These results do not establish public timing, memory, graphics, BIOS, or commercial-game compatibility.
|
||||
The pinned public memory ROM also passes in both builds, but its video-byte assertions and mode selection have documented limits.
|
||||
Separate original CPU tests check exact video-byte results across supported modes, mirrored addresses, byte lanes, and boundaries.
|
||||
They require unchanged nonzero sentinels for ignored writes and unchanged adjacent halfwords for duplicated writes.
|
||||
They also check forced blank, full source-value preservation, CPU halfword readback, nominal data costs, and device progress.
|
||||
Halfword/word stores remain writable in object regions. No emulator behavior change was needed for these tests.
|
||||
These results do not establish public timing, graphics, BIOS, save, or commercial-game compatibility.
|
||||
|
||||
ROM-runner tests use temporary files containing original instructions, without Nintendo assets or a desktop session.
|
||||
They check file-size boundaries, bounded reads, short reads, I/O errors, paths, symlinks, and read-only loading.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue