test(bios): integrate pinned BIOS read-protection baseline
Add --suite bios with hash-verified upstream files and the ARM evaluation checkpoint at 0x08000248. Preserve the r12=0 assertion, original BIOS boot, and all test bytes. Add synthetic adapter and CLI regressions. On Darwin arm64 the public ROM fails test 1 in debug and release after 64 steps and 543 nominal cycles. Document that this tests protected reads of Nintendo firmware opcodes, not general BIOS service compatibility. Online and offline preparation agree; 39 Python tests pass. Core behavior is unchanged.
This commit is contained in:
parent
661f00d682
commit
6e736c634b
11 changed files with 143 additions and 8 deletions
|
|
@ -14,7 +14,7 @@ The repository is a Cargo workspace with three crates:
|
|||
| Path | Purpose |
|
||||
| --- | --- |
|
||||
| `flake.nix`, `flake.lock` | Pinned Rust/Python development environment |
|
||||
| `tools/prepare_gba_tests.py`, `tools/gba-tests-{arm,thumb,memory}.lock.json` | Hash-verified public test source/ROM preparation and suite generation |
|
||||
| `tools/prepare_gba_tests.py`, `tools/gba-tests-{arm,thumb,memory,bios}.lock.json` | Hash-verified public test source/ROM preparation and suite generation |
|
||||
| `tools/test_prepare_gba_tests.py` | Offline adapter tests using original synthetic data |
|
||||
| `.envrc` | Automatic environment loading through direnv |
|
||||
| `AGENTS.md` | Short orientation for coding agents: crate map, commands, and where docs live |
|
||||
|
|
|
|||
74
docs/public-bios-tests.md
Normal file
74
docs/public-bios-tests.md
Normal file
|
|
@ -0,0 +1,74 @@
|
|||
# Pinned public BIOS read-protection test
|
||||
|
||||
The headless runner executes the unmodified BIOS test ROM from [jsmolka/gba-tests](https://github.com/jsmolka/gba-tests).
|
||||
It currently fails test 1 on Darwin arm64 in debug and release builds.
|
||||
This ROM tests protected BIOS reads against Nintendo firmware opcode values, not the complete BIOS service set.
|
||||
Our runner uses the original replacement BIOS. It neither supplies nor loads Nintendo firmware.
|
||||
|
||||
## Prepare and run
|
||||
|
||||
```sh
|
||||
direnv exec . python3 tools/prepare_gba_tests.py /tmp/gba-public-bios --suite bios
|
||||
direnv exec . cargo run --locked --release -- --test-suite /tmp/gba-public-bios/suite.json > /tmp/gba-public-bios-report.json
|
||||
```
|
||||
|
||||
`bios/bios.gba` is a public test cartridge, not a BIOS image.
|
||||
Use a new directory outside the repository or under ignored `roms/`. Its parent directory must exist.
|
||||
The adapter checks every selected file's size and SHA-256 digest before creating output.
|
||||
It retains the upstream license, source, ROM, `suite.json`, and `source-lock.json` without modifying them.
|
||||
Downloaded binaries and header sources must not enter tracked files.
|
||||
|
||||
Offline preparation uses the same checks:
|
||||
|
||||
```sh
|
||||
direnv exec . python3 tools/prepare_gba_tests.py /tmp/gba-public-bios-offline --suite bios --source-dir /path/to/gba-tests
|
||||
```
|
||||
|
||||
ARM remains the default suite. The Thumb and memory selections remain unchanged.
|
||||
Preparation is not a passing result. The BIOS run currently exits nonzero and writes a failed assertion report.
|
||||
|
||||
## Provenance and checkpoint
|
||||
|
||||
- Repository: `jsmolka/gba-tests`.
|
||||
- Revision: `a7113b67e63f83a9b321696ddd7042ccfad6c881`.
|
||||
- Upstream project license: MIT, copyright Julian Smolka; retained with downloaded files.
|
||||
- Test ROM: `bios/bios.gba`, 1,908 bytes.
|
||||
- SHA-256: `9d7b369fa1aa661ff03692b3d79c6f644b623d72983d0fc890e6d87a0409a3c9`.
|
||||
- Selected file hashes: [`tools/gba-tests-bios.lock.json`](../tools/gba-tests-bios.lock.json).
|
||||
|
||||
The adapter uses the upstream committed binary. No FASMARM rebuild or source-to-binary reproduction was performed on Darwin.
|
||||
The source initializes r12 to zero. Each failure sets r12 to its test number and branches to `eval`.
|
||||
Successful execution reaches the same label with r12 still zero.
|
||||
The source exits and binary ARM branch targets were checked against `eval` at `0x08000248`.
|
||||
Its first instruction is `0xe92d0003`, the shared `m_vsync` register save.
|
||||
There is another `m_vsync` inside test 3 at `0x08000194`; that is not the completion checkpoint.
|
||||
|
||||
The manifest requires ARM state at `0x08000248` and `r12 == 0`, within 1,000,000 successful machine steps.
|
||||
The budget includes original BIOS boot. All test setup executes normally; only final result rendering is excluded.
|
||||
Test 3's VBlank wait and IRQ callback are part of the test and are not bypassed.
|
||||
Preparation verifies the checkpoint word in addition to whole-file hashes.
|
||||
Timeouts, diagnostics, STOP, and nonzero r12 remain failures under the [normal runner rules](rom-tests.md).
|
||||
|
||||
## Observed result on Darwin arm64
|
||||
|
||||
Before BIOS read protection:
|
||||
|
||||
- Both builds reach `eval` after 64 successful steps and 543 nominal cycles.
|
||||
- The CPU is in System/ARM state, with CPSR `0x2000001f` and r12 = 1.
|
||||
- The report says `reason: assertion_failed`, with no emulator diagnostic.
|
||||
- Test 1 reads address zero after boot. It expects `0xe129f000`, a Nintendo BIOS opcode.
|
||||
- The emulator instead exposes the replacement BIOS reset-vector instruction at address zero.
|
||||
- The later post-SWI and IRQ checks are not reached.
|
||||
|
||||
This reveals a missing bus-protection rule and a separate firmware-specific expectation.
|
||||
Implementing protection must not substitute Nintendo opcode constants merely to pass this ROM.
|
||||
The original BIOS has a different layout, so its retained prefetch data can differ even with protection implemented.
|
||||
No public BIOS pass is claimed. The ARM, Thumb, and memory suites remain separate passing results.
|
||||
Exact fetch-pipeline behavior and full BIOS compatibility remain unverified.
|
||||
|
||||
## References
|
||||
|
||||
- [Pinned BIOS test source and expected opcodes](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/bios/bios.asm).
|
||||
- [Shared initialization and result macros](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/lib/macros.inc).
|
||||
- [GBATEK BIOS read protection](https://problemkaputt.de/gbatek-gba-unpredictable-things.htm).
|
||||
- [Pinned license](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/LICENSE).
|
||||
|
|
@ -68,6 +68,7 @@ Hardware references used for the core:
|
|||
- [Pinned jsmolka Thumb entry](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/thumb.asm), [branch tests](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/branches.asm), and [memory tests](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/thumb/memory.asm), reviewed for r7 failure IDs, success-marker reset, the Thumb-to-ARM evaluation bridge, and empty-list transfer expectations.
|
||||
- [Pinned jsmolka memory entry](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/memory/memory.asm), [mirror tests](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/memory/mirrors.asm), and [video-byte tests](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/memory/video_strb.asm), reviewed for the r12 checkpoint, binary exit targets, limited byte-write assertions, and mode-selection caveats.
|
||||
- [gbadoc memory layout](https://gbadev.net/gbadoc/memory.html), consulted for VRAM mirror ranges alongside GBATEK's video-memory layout and byte-write rules.
|
||||
- [Pinned jsmolka BIOS test](https://github.com/jsmolka/gba-tests/blob/a7113b67e63f83a9b321696ddd7042ccfad6c881/bios/bios.asm), reviewed for the r12 evaluation checkpoint, internal VBlank wait, and Nintendo-firmware-specific protected-read expectations after boot, SWI, and IRQ.
|
||||
- [armwrestler-gba-fixed](https://github.com/destoer/armwrestler-gba-fixed) and [arm7wrestler](https://github.com/Arisotura/arm7wrestler), considered as test sources; not integrated.
|
||||
- [SingleStepTests ARM7TDMI](https://github.com/SingleStepTests/ARM7TDMI), considered as an experimental, emulator-generated test source; not integrated.
|
||||
- [GBATEK ARM single data transfers](https://problemkaputt.de/gbatek-arm-opcodes-memory-single-data-transfer-ldr-str-pld.htm), for address calculation, indexing, load extension, and nominal timing.
|
||||
|
|
|
|||
|
|
@ -136,7 +136,8 @@ The first public integration is [jsmolka/gba-tests ARM](public-arm-tests.md), wi
|
|||
The pinned ARM ROM now passes this checkpoint on Darwin arm64. Historical failures remain documented.
|
||||
The [pinned Thumb ROM](public-thumb-tests.md) also passes its verified r7 checkpoint after the empty-list store correction.
|
||||
The [pinned memory ROM](public-memory-tests.md) also passes, without a core change and with explicit coverage limits.
|
||||
Next, integrate a pinned public BIOS suite with a verified result protocol.
|
||||
The [pinned BIOS read-protection ROM](public-bios-tests.md) reaches its checkpoint but fails test 1 with the original replacement BIOS.
|
||||
Its expected values depend on Nintendo firmware. Record that failure rather than changing the assertion or returning guessed constants.
|
||||
For further public ARM7TDMI/GBA tests, inspect their source, license, entry assumptions, and result protocol.
|
||||
Pin source revisions and build instructions before comparing emulator changes.
|
||||
Adapt suites to verified completion addresses and result locations; do not guess them or treat timeouts as passes.
|
||||
|
|
|
|||
|
|
@ -69,6 +69,7 @@ Update this file when a feature lands or a limit is removed.
|
|||
- Headless JSON ROM suites with per-case checkpoints, register/CPSR/memory assertions, bounded budgets, and structured failure reports.
|
||||
- Hash-pinned preparation of public `jsmolka/gba-tests` ARM, Thumb, and memory ROMs, with verified result-register checkpoints.
|
||||
- These pinned ROMs pass their checkpoints on Darwin arm64 in debug and release builds; coverage limits remain documented.
|
||||
- A pinned public BIOS read-protection ROM has a verified checkpoint and an explicit failing baseline with the original BIOS.
|
||||
- ARM test/compare Rd=15 status restoration, without a PC result write or nominal refill.
|
||||
- ARM unused-memory data reads return a PC+8 word snapshot, with normal byte lanes and load rotation.
|
||||
- Little-endian byte, halfword, and word reads/writes.
|
||||
|
|
@ -119,6 +120,7 @@ The [ROM suite runner](rom-tests.md) has original regressions and a [pinned publ
|
|||
The pinned public ARM ROM passes its result checkpoint in debug and release builds.
|
||||
The [pinned public Thumb ROM](public-thumb-tests.md) also passes after correcting its empty-list store PC value.
|
||||
The [pinned public memory ROM](public-memory-tests.md) passes without a core change; its video-byte assertions have documented limits.
|
||||
The [pinned BIOS read-protection ROM](public-bios-tests.md) fails test 1; it expects Nintendo firmware opcode values.
|
||||
Other public suites remain unverified. These results do not establish full CPU or memory compatibility.
|
||||
The runner requires known completion addresses and has no debug-port protocol, scripted input, or rendered-image assertions.
|
||||
|
||||
|
|
@ -132,7 +134,7 @@ Writes to cartridge addresses return errors instead of modeling cartridge hardwa
|
|||
|
||||
## Next steps
|
||||
|
||||
1. Add a pinned public BIOS suite with a verified result protocol, then address its first compatibility failure.
|
||||
1. Add BIOS read protection based on supplied firmware instructions, not hard-coded values from the public BIOS test.
|
||||
Extend open-bus support with verified Thumb region/alignment rules and bus history, rather than a blanket unmapped-read fallback.
|
||||
2. Refine nominal timing with a fetch pipeline, Game Pak prefetch, per-access device updates, and verified timer/IRQ delays.
|
||||
3. Validate keypad retrigger behavior with hardware tests and add remaining device registers; extend BIOS reset coverage as sound/serial support becomes available.
|
||||
|
|
|
|||
|
|
@ -52,6 +52,7 @@ Additional tests cover DMA/IRQ accounting, HALT/STOP, diagnostic state, case iso
|
|||
These original tests validate the runner. The [public ARM adapter](public-arm-tests.md) provides a separate passing checkpoint result for its pinned ROM.
|
||||
The [public Thumb adapter](public-thumb-tests.md) uses a separate lock and r7 assertion after its return to ARM state.
|
||||
The [public memory adapter](public-memory-tests.md) uses its own lock and an ARM-state r12 checkpoint.
|
||||
The [public BIOS adapter](public-bios-tests.md) also uses an ARM-state r12 checkpoint, but currently reports test 1 as failed.
|
||||
Preparation tests use only synthetic original bytes and no network:
|
||||
|
||||
```sh
|
||||
|
|
@ -59,7 +60,7 @@ direnv exec . python3 -B -m unittest discover -s tools -p 'test_*.py' -v
|
|||
```
|
||||
|
||||
They verify suite selection, pinned URLs, hashes, sizes, checkpoint validation, offline copies, destination safety, and non-overwriting output.
|
||||
Thumb cases also verify the return-to-ARM bridge. Command-line tests cover default ARM selection, explicit Thumb/memory selection, and errors.
|
||||
Thumb cases also verify the return-to-ARM bridge. Command-line tests cover default ARM selection, explicit Thumb/memory/BIOS selection, and errors.
|
||||
Compare/status CPU regressions verify saved flags/masks, bank switching, sequential PC/timing, User/System fallback, and atomic diagnostics.
|
||||
Load-alias regressions cover widths, signed/unaligned values, immediate/register offsets, pre/post-indexing, every non-PC register bank, and address wrapping.
|
||||
They verify loaded-value precedence, preserved flags, unchanged stores, nominal timing, I/O reads before device progress, deferred IRQ delivery, and atomic errors.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue