Booz Allen Cyber Weapon Index: 18 Models, One Full Kill Chain

The headline finding of Booz Allen's new index is not the ranking. It is that a cheap piece of software can erase the ranking. The US defense consultancy set 18 AI models loose as autonomous attackers on a real enterprise network; exactly one, Anthropic's Claude Mythos, completed the full kill chain. Yet Claude Sonnet 5, ranked 15th, came close to Mythos once it was wired into an off-the-shelf attack harness. In the report's own words: the model is no longer the unit of risk. The system is.

How the test was built

The Cyber Weapon Index evaluated nine US and nine Chinese models under identical conditions. The environment was a defended, production-grade Active Directory network; the goal was to progress from initial access to full domain admin. Models got no curated tool menu and no extra scaffolding. Scoring relied on network telemetry, host logs, domain controller data and intrusion-detection sensors rather than on what the model claimed to have done. A second component measured vulnerability research on compiled software with no source code. Scenarios ran both with and without stolen credentials.

The ranking

Scores out of 80, as reported by The Register on September 2: Claude Mythos 80, Grok-4.5 49, GPT-5.6 Sol 46, Muse Spark 1.1 and Kimi K3 38, GLM-5.2 37, Claude Opus 4.8 36, GPT-5.5-Cyber 34, Nemotron-Ultra 33, DeepSeek-V4-Pro 23, DeepSeek-V4-Flash and Qwen3.5-397B 17, MiniMax-M3 and Nemotron-Super 15, Claude Sonnet 5 13, GLM-4.5-Air 11, Qwen3.6-35B 9, Qwen3-Coder 4.

The spread: one model completed the chain, three more (Grok-4.5, Muse Spark 1.1, GLM-5.2) reached full domain control, four achieved lateral movement, two stopped at credential access. All but one gained initial access. Given credentials, Mythos reached admin on every attempt; without them, it still took the domain.

Three findings, one conclusion

• Real vulnerability discovery is still the dividing line. Most models made progress against deliberately planted flaws; Mythos was the only one to exploit genuine, previously unseen vulnerabilities.
• The harness moves the needle more than the model. Sonnet 5 scored 13 alone and closed most of the 67-point gap to Mythos when paired with inexpensive software that connects a model to hacking tools. Booz Allen's phrasing: the harness matters "as much as, or more than, the model itself."
• Guardrails are not fixed properties. The same model's safety behavior shifted with configuration and context. Capability is not country-specific either; Chinese models sit inside the top ten.

The forecast is blunt: most models will be able to run the full chain within six months. The recommendations: enforceable, sector-specific containment deadlines for critical infrastructure operators; a national program that tests foreign and open-weight models under realistic conditions; governed access for vetted defenders.

What this means for a company that is not a defense contractor

The thesis is simple. On the defensive side, "who has access to the strongest model" is now a secondary question. An attacker who wraps a mid-tier open model in a good harness can move laterally inside your network, which makes cutting response time from hours to minutes mandatory rather than aspirational. Concretely: harden privileged accounts in Active Directory, rehearse the stolen-credential scenario in a tabletop exercise, and verify that your lateral-movement detection rules actually fire. Apply the same logic when buying AI products: audit the system built around the model, not the model's name.

Sources: Booz Allen: Cyber Weapon Index, The Register, The Next Web