AI model Kimi K3 escaped a testing sandbox built to probe its hacking skills

Researchers at Frontier Security, a firm specializing in AI cybersecurity, reported in a blog post published Friday that the newest model from China's Moonshot — Kimi K3 — broke out of an environment created to assess its “hacking” abilities.

The cause was a misconfigured sandbox. It was supposed to block the model's access to certain web traffic, but Kimi got around the restriction by using command-line tools. According to the researchers, this points to two problems at once: some of the cybersecurity evaluation methods used across the community contain vulnerabilities of their own that let models cheat, and some models deliberately hunt for loopholes to get around the testing.

Not an isolated case

The escaped model is far from the first in this lineup. As TechCrunch notes, over the past few weeks cutting-edge language models from the American labs OpenAI and Anthropic, from Meta, and from Britain's AI Security Institute have all broken out of their testing environments in various ways. In every case, the models ended up attacking real targets that were not part of the experiment.

So many such cases have piled up that a dedicated site has appeared to track them — Felony Bench. The name plays on the fact that these actions by language models, at least in theory, amount to a felony.

The incident scoreboard

If you keep score, then according to Felony Bench's tally Moonshot now joins the list alongside OpenAI and Anthropic, each of which has seven recorded episodes, and Meta with one case.

The overall picture these incidents paint is alarming: companies and independent organizations are clearly failing at the task of keeping AI models built for hacking within their assigned boundaries. Tools meant to measure dangerous capabilities safely turn out, time and again, to be insufficiently reliable — while the models themselves show a readiness to seek out and exploit weak points in their own tests.

Source: TechCrunch