Opus 5 Is More Resistant to Prompt Injection

Bruce Schneier flagged a chart showing how language models fare against prompt injection attacks — the smuggling of malicious instructions into a prompt. The chart draws on the IPI benchmark, which measures an attacker's probability of success.

Claude's Progress

Opus 5 noticeably improved on its predecessor, Opus 4.8: the odds of a successful attack within 15 attempts dropped from 5.5% to 2.0%, and for a single attempt from 0.5% to 0.2%. The new model also outdid others in the same family: Sonnet 5 (5.9% at 15 attempts) and Mythos 5 (2.6%). In the end, Opus 5 turned out to be the most resistant of the models tested.

A Gap Over the Competition

Opus 5 also surpassed every model from outside developers. The toughest of the “outsiders” was Muse Spark — 16.5% within 15 attempts, more than eight times worse than Opus 5's result.

The most powerful variant of GPT 5.6, named Sol, was barely distinguishable from its predecessor GPT 5.5: 20.0% versus 20.8% over 15 attempts. That's ten times higher than the 2.0% posted by Claude Opus 5. The other GPT 5.6 variants held up even worse — Terra came in at 30.4%, and Luna at a full 43.9%. One detail is telling: a single attempt against GPT 5.6 Sol succeeded 3.1% of the time — more than the 2.0% an attacker managed against Opus 5 only after fifteen attempts.

A Cautious Conclusion

Schneier reminds us that, in the general case, prompt injection cannot be fully prevented — it's a fundamental limitation. In specific scenarios, however, defenses are becoming ever more effective, and the latest figures bear that out.

Source: Schneier on Security