DeepSeek-V4.1 Flash: Squeezing the KV Cache Fourfold for Long Context

The blogger zartbot works through the technical report on DeepSeek-V4.1-Flash and immediately confesses to a bit of bewilderment: at first he took the release for a run-of-the-mill update, but a speed approaching 420 tokens per second and rumors about retiring the entire V4 Pro line made him take a closer look. After reading the full report, he half-jokingly concludes that, given the depth of the architectural overhaul, the model would have deserved the name DeepSeek-V5 Flash.

Why all this

The main motivation is stated right in the report's title — to push KV-cache compression to the limit. The reason is that “long-lived” agentic scenarios balloon the context to enormous sizes, while frequent tool calls put a heavy load on the prefill stage. The key-value cache, meanwhile, fills up both HBM and external SSDs, and shuttling it around runs into the bandwidth of the interconnects. It is precisely the storage and movement of data, rather than the computation itself, that the author sees as the bottleneck when serving long sequences.

What kind of model this is

DeepSeek-V4.1-Flash is a multimodal mixture-of-experts model with 552 billion parameters, native image support, and a context of up to a million tokens. At the same time, only about 8 billion parameters are activated per token during prefill and 16 billion during decoding — savings that are especially noticeable in agentic tasks, where the input dominates the output.

At its core lies a Causal Encoder-Decoder architecture: a 40-layer network is split into an “encoder” and a “decoder” of 20 layers each, and the decoder's global cache is obtained by projecting the encoder's final states. Thanks to this, most positions of a long query pass only through the first 20 layers, which cuts prefill computation roughly in half. The idea traces back to the YoCo architecture (“cache once”).

Three dimensions of compression

The cache savings themselves come from the CSA2 mechanism, which squeezes the data along three axes at once. Along the channels, a shared 512-dimensional latent vector replaces the separate keys and values of each attention head. Along the sequence length, the encoder merges two neighboring positions into a single entry. Along the layers, several layers share a single global KV — leaving just three copies of the encoder cache and one of the decoder for the whole network. By adding quantization in the FP4 format, the developers brought the storage footprint down to roughly 890 bytes per token.

All told, according to the report, at equal sequence length the working KV cache takes up about a quarter, and the permanently stored one about an eighth, of the previous DeepSeek-V4-Flash — while the model's overall quality is claimed to be even higher. A substantial part of the breakdown is devoted to the mathematical justification of why keys and values can be “frozen” and reused across layers, rewriting only the query Q — the author interprets such a construction as a recursive transformer and draws an extended analogy with a processor's cache hierarchy.

Source: Hacker News