A Nix binary cache is just three functions — which means anything can be one

While digging into a project of his own, Farid Zakaria stumbled onto a curious observation: to become a Nix binary cache, a server doesn't need to know anything about Nix. All it has to do is answer three requests — serve the nix-cache-info file with its StoreDir line, hand back a .narinfo by hash with the archive's metadata, and finally return the compressed archive itself from the address in the URL field. That's it. Any static file server is up to the task.

Hence the author's provocative thesis: if a humble file server can fill the role of a cache, then almost anything can be made into one. Take GitHub Pages, for instance, which automatically sets the permissive access-control-allow-origin: * header — just commit the output of nix copy --to file:// into a repository, and your free substituter is ready to go.

Why this is safe

The secret is that Nix doesn't trust the transport. The signature in the .narinfo covers only StorePath, NarHash, NarSize, and References — not the URL, not the file size, not the compression method. After it's downloaded, the archive is unpacked and the client checks the NarHash. That's exactly why a package signed by cache.nixos.org can be downloaded from anywhere and the signature still stays valid. The URL field isn't even required to point at the same host, or even the same protocol.

A cache out of anything

The author assembled a whole collection of curious implementations. gachix stows archives in a git object database, which deduplicates and compresses the data on its own — a saving of roughly 82% over an ordinary cache. There's a proof-of-concept that scatters .narinfo files and four-kilobyte chunks of the archive across DNS TXT records. A cache can be a pastebin (with expiration acting as a natural garbage collector), an OCI registry, or even YouTube — the “infinite storage glitch” project encodes data inside a video clip.

npm as a substituter

npm gets a section of its own. Since nix copy --to file:// creates a directory and npm knows how to publish directories, the author patched the classic hello to print “Hello from the npm registry!”, signed the closure with his own key, and published the package. After that Nix happily pulls the whole closure from unpkg.com and runs the binary. As a bonus, you get “channels” via npm dist-tags: the latest tag is mutable, while specific versions are immutable.

The main drawback is the lack of incremental publishing: fifty closures that share a glibc will upload glibc fifty times. In theory this could be cured by publishing each store path as a separate package with an index, but the author honestly admits he won't build such a thing — toward the npm ecosystem, that would be less than gentlemanly.

Source: Lobsters