A computer in the weights: a working RISC-V CPU compiled into Qwen 3.5
Translated by a model; the original is in Russian.
Divyansh Agrawal has unveiled Qwenputer, an experimental model. It is a stock Qwen 3.5 4B with one addition: a RISC-V CPU embedded in its weights, which uses the model's own Gated DeltaNet (GDN) attention layers as RAM. The processor runs entirely within the model's forward passes. It handles arithmetic as well as C and MicroPython code, with no external runtime. The author stresses that this is not a plugin, a tool or a sandbox.
An RNN as exact memory
In Qwen 3.5, three out of four attention layers use Gated DeltaNet, a linear recurrent network. Each head keeps a state matrix that is updated by the delta rule. Agrawal set the forgetting gate to 1, so nothing fades, and the write strength (beta) to 1, so old values are fully replaced. He also used one-hot keys. As a result, every write overwrites exactly one row of the state, and every query reads back an exact value, much like accessing a memory cell by address. This memory is cheaper than ordinary attention: a GDN head's state does not grow with the length of the conversation, unlike the KV cache.
The idea that the delta rule acts as an associative memory is not new; Schlag, Irie and Schmidhuber described it back in 2021. What is new is making that memory exact and addressable, inside a production model, and reliably enough to run a whole CPU on it.
A CPU without training
No training was involved: a dedicated compiler sets every weight directly. The CPU is described as a circuit of small steps such as "read a register" or "jump if zero". Attention heads perform lookups, feed-forward units do the arithmetic, and DeltaNet heads hold the data. Each instruction becomes a few generated tokens: a row address (ADDR), a value (VAL) and a closing STEP. While the model is generating these tokens, the CPU is actually running.
The core implements RV32IM with the Zba extension and a small unit for big-number arithmetic. It occupies 11 layers in front of Qwen's 32, passes the official riscv-tests, matches qemu's results, and has executed 18 million consecutive instructions without a single error. It is still a long way from being able to run Linux.
Qwen talks to the CPU through the familiar tool-call format, with no post-training. The original weights are untouched: the computer weights sit on top, in extra dimensions and layers. A small driver, RvChat, loads the firmware.
Results
On DeepMind's sets, Qwenputer scores 97.5% on big-number division (72.5% for the base model, 95% with a Python tool) and 97.5% on multiplication (55% and 100% respectively). On NUPA's long-integer arithmetic, accuracy rises from 48.6% to 100%. Checking roughly 230,000 output positions against a reference machine turned up zero mismatches. The model's ordinary text is practically unchanged.
A hard drive, too
A CPU apparently wasn't enough, so the author added an experimental filesystem. File names are hashed into one of 1,024 directory slots kept in the DeltaNet state. It holds up to 512 files, or about 53,000 tokens. The state can be saved and loaded into the next conversation. With this memory, the model gets facts that changed between chats right 90% of the time. Simply pasting the old chats back into context yields 30%.
Limitations
The checkpoint weighs 18.8 GB, roughly double the base model, though it is mostly zero padding. The model cannot run without the RvChat driver, and it is slow: the batched executor manages about 3,200 CPU tokens per second on an L4. Next, the author wants Qwen to run Python on the built-in MicroPython and to speed up the CPU. He also sees the project as a testbed for AI safety research. The model runs on a single 24 GB GPU, and the code and weights are open.
Source: Hacker News
Comments
Sign in, to comment.