Nadella says we should treat every AI model as ‘compromised’

Microsoft CEO Satya Nadella has published a lengthy post on X laying out his views on the dangers posed by highly advanced AI models and how to confront those risks.

Nadella argues we can no longer accept a world where AI is treated as a "set of nested black boxes" whose advice and actions we simply take or leave. He calls instead for a more transparent system in which models can be contained and observed, and which leaves behind "tamper-proof human readable evidence" of how they operate.

A familiar checklist, with one addition

Much of what he recommends echoes what others in the industry have already said: timely incident disclosure, independent audits, verifiable data, and containment. It is on that final point that Nadella appears to go a little further than most of his peers.

"We must assume a model is compromised and contain it from the start," he writes. "Think of it like an emergency brake. An authorized person should always be able to pause or shut down a model mid-task. More advanced models will require more advanced containment technologies that we need to standardize on."

One quirk, though: throughout the post Nadella insists on calling AI "super intelligence" — a label that does little to make his sober recommendations sound more measured.

Source: The Verge