Run NES Games Inside a Compiled Qwen3.5 Model with nesformer

What is nesformer?

nesformer is an experimental NES runtime in which the console's 6502 CPU, 2 KB of work RAM, and cartridge program ROM operate inside the weights and state of a standard Qwen3_5ForCausalLM checkpoint. The weights were not trained. Instead, a compiler calculated them to implement the CPU circuit.

During a game, the model predicts the CPU execution log one token at a time. A Python host program supplies the remaining console components: it renders graphics, reads controller input, handles timing, answers I/O reads, and injects the vertical-blanking NMI interrupt.

The project builds on Qwenputer and reuses its compiler and central memory technique. Each Gated DeltaNet head maintains a state matrix. One-hot keys and queries allow rows in that matrix to behave as memory cells, holding registers, RAM, and cartridge data.

Key features

• Compiled rather than trained: every nonzero weight is generated from a circuit specification.
• Standard checkpoint format: the checkpoint is an ordinary Hugging Face folder and can be loaded with Qwen3_5ForCausalLM.from_pretrained without custom model code.
• Complete official 6502 instruction set: the circuit implements all 151 official opcodes, addressing modes, flags, stack operations, BRK, and NMI entry.
• Reusable NROM support: the same weights can run different iNES mapper 0 ROMs because the selected cartridge is written into model state at startup.
• Scripted controls: button presses can be scheduled by frame from the command line.
• Visual exports: runs can be displayed in a compatible terminal and saved as GIF or PNG frames.
• Execution verification: the check command compares model output with the bundled reference CPU and stops at the first difference.
• Multiple execution engines: nesformer offers a fast numba engine, a plain NumPy engine, and a stock Transformers engine.

Understand what runs inside the model

The checkpoint contains the 6502 core, CPU registers and flags, 2 KB of work RAM, and a fixed 32 KB program-ROM table. It also handles program fetches, data reads, stack operations, and OAM DMA reads.

The host remains responsible for the PPU renderer, palette, controllers, vblank NMI timing, cycle counting, and APU register handling. Sound is not implemented. The host injects values for I/O reads and NMI events into the token stream, but it does not directly inspect the model's registers or RAM outside the emitted trace.

nesformer emulates the CPU through model inference, but it is not a complete NES implemented entirely inside the weights.

Requirements and folder setup

You need Python 3.11 or newer, the nesformer source archive, and the separate weights archive. Extract both archives into the same top-level folder. The weights directory must sit beside the Python package rather than inside it.

nesformer/
├── nesformer/
├── chipformer/
├── roms/
├── examples/
├── tests/
├── pyproject.toml
└── weights/
    ├── config.json
    ├── generation_config.json
    ├── model.safetensors
    ├── nesformer.json
    └── SHA256SUMS

Run all commands from the top-level directory shown above. By default, the CLI looks for weights/ beside the nesformer/ package.

Verify the downloaded weights

Use the included checksum file to confirm that the weights were extracted correctly:

shasum -a 256 -c weights/SHA256SUMS

If you keep the checkpoint elsewhere, pass its location with --weights /path/to/weights when using run, check, or trace. The info command accepts the checkpoint path directly.

Install nesformer

Create and activate a virtual environment, then install the required numerical and image-processing packages:

python3 -m venv .venv
source .venv/bin/activate
pip install numpy scipy numba pillow

On Windows, activate the environment with:

.venv\Scripts\activate

The default fast engine uses numba. Its first run spends roughly 20 seconds compiling the engine. Later runs reuse that compilation work.

Run your first game

The repository includes freely licensed test ROMs. The following command runs Thwaite for 600 frames, displays frames in the terminal, and supplies a scripted sequence of Start and A button presses:

python -m nesformer run roms/thwaite.nes --frames 600 --show \
  --buttons "start@90-95+200-205,a@230-233+250-253+270-273+290-293+310-313+330-333+350-353+370-373+390-393+410-413+430-433+450-453+470-473+490-493+510-513+530-533+550-553+570-573+590-593"

The --show option draws each generated frame in a terminal with 24-bit color support, such as iTerm2 or Windows Terminal. When execution finishes, the CLI saves an animated GIF named after the ROM, such as thwaite.gif.

Performance is much slower than a physical NES. After the initial compilation, a frame generally takes about one to six seconds. A 600-frame run represents 10 seconds of NES time and may take around 25 minutes.

Run your own compatible ROM

Place your legally obtained ROM in the roms/ directory or provide another path. The ROM must use the NROM board, identified as iNES mapper 0. Compatible examples include Super Mario Bros., Donkey Kong, and Excitebike, although these commercial ROMs are not included.

This example starts Super Mario Bros., holds Right, and schedules several jumps:

python -m nesformer run roms/smb.nes --frames 600 --show \
  --buttons "start@60-65,right@200-600,a@250-265+300-315+360-375+420-435+480-495+540-555"

The ROM is loaded into the model's DeltaNet state at startup. You do not need to rebuild the checkpoint for each compatible game.

Script controller input

The --buttons option accepts comma-separated entries. Each entry contains one or more button names, an @ symbol, and one or more frame ranges.

start@60-65
right@200-600
a@250-265+300-315
right+b@200-600

• start@60-65 presses Start from frame 60 through frame 65.
• right@200-600 holds Right for the specified interval.
• a@250-265+300-315 schedules two A-button presses.
• right+b@200-600 holds Right and B simultaneously.

Available names are a, b, select, start, up, down, left, and right. Frames start at power-on and advance at 60 frames per second. Frame 600 therefore corresponds to 10 seconds of NES time.

It is usually safe to begin a press slightly early because games ignore controls on screens where movement is unavailable. Button duration can matter; for example, a longer A press produces a higher jump in Super Mario Bros.

Control frame output

By default, run executes 120 frames and writes a GIF named after the ROM. Use output options to customize the result:

python -m nesformer run roms/thwaite.nes \
  --frames 300 \
  --every 2 \
  --scale 3 \
  --gif output/thwaite.gif

• --frames N stops after N frames.
• --gif PATH selects the animated GIF destination.
• --png DIR saves retained frames as individual PNG files.
• --every N retains every Nth frame.
• --scale N enlarges saved frames by an integer factor.
• --show displays generated frames in the terminal.

Using --png by itself disables GIF output. Add an explicit --gif option when you want both PNG frames and an animation.

python -m nesformer run roms/thwaite.nes \
  --frames 120 \
  --png frames \
  --gif thwaite.gif

Inspect and verify execution

Compare against the reference CPU

The check command runs the checkpoint and the project's conventional reference 6502 side by side. It stops at the first token difference:

python -m nesformer check roms/thwaite.nes --tokens 50000

The default comparison length is 50,000 tokens. You can pass the same --buttons, --weights, --engine, and --no-skip options accepted by other execution commands.

Print the model's token stream

Use trace to inspect the CPU records emitted by the model:

python -m nesformer trace roms/thwaite.nes --tokens 100

The default trace length is 40 tokens. Most CPU instructions require one token, while instructions involving I/O, pointer reads, stack operations, or DMA may produce multi-token records.

Save token IDs during a run

The run command can write generated token IDs to a file for later analysis:

python -m nesformer run roms/thwaite.nes \
  --frames 60 \
  --trace trace.txt

You can also use --max-tokens N to stop execution after a fixed number of generated tokens instead of relying only on a frame limit.

Choose an execution engine

nesformer provides three engines with different performance and dependency profiles:

• fast: the default numba implementation. The README reports about 1,650 tokens per second on one CPU core of the development machine.
• numpy: a simpler NumPy implementation reported at roughly 210 tokens per second.
• hf: stock Hugging Face Transformers, reported at approximately 10 tokens per second.

Select an engine with --engine:

python -m nesformer run roms/thwaite.nes --frames 3 --engine numpy

Use stock Transformers

Install the additional dependencies before selecting the Hugging Face engine:

pip install "torch>=2.6" "transformers>=5.22"
python -m nesformer run roms/thwaite.nes --frames 3 --engine hf

The checkpoint needs no custom model implementation. Internally, nesformer loads it, decodes one padding token, writes the initial machine state into the cache's DeltaNet states, and then feeds generated tokens back one at a time.

Advanced tips

Keep weights outside the repository

Use --weights when the checkpoint is stored in another location:

python -m nesformer run roms/thwaite.nes \
  --weights /path/to/weights \
  --frames 120

Inspect checkpoint metadata

Use the information command to describe the checkpoint:

python -m nesformer info

If the weights are not in the default directory, provide their path to info.

Rebuild the checkpoint

The repository includes the compiler needed to regenerate the circuit checkpoint. A normal build writes to weights/ and takes about 10 seconds according to the README:

python -m nesformer build

Choose another output directory with:

python -m nesformer build --out compiled-weights

Understand idle-loop skipping

NES games frequently wait for vblank in tight loops. When the host detects an exactly repeating loop with no RAM or I/O writes, it advances the clock close to the next PPU event and then runs the remaining iterations. This optimization substantially reduces token generation while preserving visible frames and I/O timing.

Pass --no-skip to execute every idle-loop iteration. This is slower, but it can be useful when studying the complete token sequence:

python -m nesformer trace roms/thwaite.nes --tokens 100 --no-skip

Run the test suite

Install the testing dependencies and run pytest:

pip install pytest py65
python -m pytest -q tests

The tests take a few minutes. They compare the reference CPU with the nestest log, check the compiled circuit against the reference, verify free-running weights, and test the idle-loop optimization.

Important limitations

• NROM only: games must use iNES mapper 0. Bank-switching mappers such as MMC1 and MMC3 are unsupported.
• Official opcodes only: encountering an unofficial 6502 opcode emits HALT.
• Execution from ROM: code must execute from cartridge ROM; jumping into RAM emits HALT.
• No IRQ support: BRK and NMI work, but APU frame and DMC IRQs do not.
• Limited I/O behavior: read-modify-write instructions on I/O registers do not produce an I/O write, and addresses from $4020 through $7FFF read as zero.
• Approximate graphics details: rendering occurs one scanline at a time, mid-scanline effects are not reproduced, and sprite-0 hit and sprite overflow are approximate.
• No audio: APU register writes are accepted but sound is not generated.
• Slow execution: even with idle-loop skipping, the project runs roughly 150 to 290 times slower than a real console.

Conclusion

nesformer demonstrates that a transformer checkpoint can serve as an exact compiled computer rather than a trained language model. Once the source and weights are arranged correctly, you can run compatible NROM games, automate controller input, export frames, inspect CPU tokens, and compare execution against a reference 6502. Its mapper, graphics, audio, and speed limitations make it an experimental system, but its standard Qwen3.5 checkpoint and verification tools provide a practical way to study computation encoded directly in model weights and DeltaNet state.