Three Open Letters on the Future of AI: Open Weights, Distillation, and “Pacing the Frontier”

Over the past few weeks, several notable open letters have appeared around the regulation of artificial intelligence in the US. Simon Willison has gathered them into a single overview.

Open Weights vs. Closed Models

The letter “Open Weights and American AI Leadership,” prepared with the participation of Microsoft and dated July 24, was signed by 235 companies, among them NVIDIA, Amazon, Y Combinator, The Linux Foundation, and later OpenAI as well. Its goal is to keep the American government from banning or restricting open-weight models under the pretext of “safety.”

The argument is simple: betting solely on closed models does not by itself guarantee safety. They too can be hacked, used for harm, or suffer failures that outside observers have no way of detecting. And concentrating cutting-edge capabilities in the hands of a few providers creates single points of failure and weakens competition. Open weights, by contrast, allow a broad community of researchers to study how models behave, find vulnerabilities, and gradually improve them.

An unexpected twist is that the letter explicitly defends distillation — that is, training one model on the outputs of another. The authors urge readers not to confuse legitimate development techniques with bad-faith appropriation, and they remind us that the tradition of learning from existing technologies and building on them goes all the way back to the open-source movement.

Anthropic's Distinctive Stance

Conspicuously absent from the signatories is Anthropic. Three days later, the company published its own response, “Our position on open-weights models.” The company's head, Dario Amodei, emphasized the risks: authoritarian states could build models more powerful than American ones, and the models themselves could be used for cyberattacks or biological attacks. He called for combating “industrial distillation,” while noting that Anthropic had never advocated a complete ban on open weights.

A Call to Slow the Race

On July 28, a third letter came out — “Pacing the Frontier,” signed by 1,324 employees of leading AI companies, including Jakub Pachocki (OpenAI), Ilya Sutskever (Safe Superintelligence), as well as Dario Amodei and Jack Clark of Anthropic. They ask the US government to support international efforts to develop the technical and governance tools that would make it possible to deliberately slow the development of automated AI.

The concern stems from the combination of fierce competition and accelerating progress driven by AI increasingly developing itself. Willison offers telling examples: Anthropic writes 80% of its code with the help of Claude Code, OpenAI's Sol system has cut service costs by 20%, and Kimi K3 designed a chip to run its own nano-model. Against this backdrop, the signatories' anxiety looks quite understandable.

Source: Simon Willison