Cybersecurity analysis: Claude Mythos Preview had a 73% success rate on expert-level capture-the-flag challenges, which no model could finish before April 2025
one from Anthropic and another from an independent gov research group. …
AI Security Institute
Context & Ripple Effects
This result marks a threshold crossing in AISI’s expert challenge set: the corpus says no model had completed these tasks before April 2025. It sits alongside Anthropic’s report that the same general-purpose system found thousands of high-severity vulnerabilities across major software platforms, making cyber capability a central part of the model’s profile rather than a narrow benchmark result.
Anthropic gains an independently framed capability signal for Mythos Preview, while AISI’s challenge suite becomes a more consequential yardstick for evaluating frontier-model cyber performance.
Organizations assessing access to Mythos Preview must treat high-end cyber-task capability as an operational governance consideration, not merely a product-quality metric.
Second-order effects
Competing frontier-model developers face pressure to demonstrate comparable performance on cyber evaluations and to show how they constrain or supervise high-risk use cases.
Security teams and model providers will have stronger incentives to pair model deployment with testing, monitoring, and disclosure processes, since capable models can support defensive vulnerability work as well as raise misuse concerns.
Third-order effects
If successive models continue clearing previously unsolved cyber ranges, evaluation results are likely to become a practical input to differentiated access controls and frontier-model release decisions.
The pattern could concentrate cybersecurity influence among the few developers and evaluators able to build, test, and govern systems at this capability level, increasing the importance of independent benchmarks and access governance.
The trend: Frontier AI is moving from isolated cyber-benchmark breakthroughs toward a competitive cycle in which advanced cyber capability and deployment controls evolve together.
claude mythos can take over a corporate network. Few immediate thoughts * doing release preview and testing breadth of capabilities and informing public is the responsible thing to do. I can see “boy who cried wolf” meme from gpt2 though. * clearly this is high value
As assessments of Mythos like UK AISI's come out, there may be a tendency to (1) breathe a sigh of relief that the capabilities are perhaps not quite as daunting as might have been (2) downplay how significant this is. But (1) this is the worst frontier AI will ever be, and it
I never understood the flurry of posts that Mythos worries were overblown and just marketing. Anthropic released the model to 50 major companies; if this was true, we'd hear chatters of disappointment pretty quickly. Instead we heard crickets. Past view days we've seen data
First external evaluation of Anthropic's claims about Mythos, from @AISecurityInst: “We conducted cyber evaluations of Mythos and found continued improvement in capture-the-flag and significant improvement on multi-step cyber-attack simulations.” [image]
This is yet another example of Claude Mythos's incredible hacking capabilities. I expect we'll see more examples and independent evaluations in the coming weeks that make clear just how powerful (and dangerous, in the wrong hands) this model could be.
The key words here are “COULD be directed to autonomously compromise SMALL, WEAKLY defended, and VULNERABLE systems if GIVEN network ACCESS.” reads skiddy level.
After AISI tested Opus 4.6 I said I thought AI models would be able complete our easiest cyber range “soon” - I didn't expect it to be the very next model. AI capabilities are increasing incredibly quickly. We must be prepared for the risks. Check your cyber security!
UK AISI has published its evaluation of Claude Mythos' cyber capabilities. It says it found “significant improvement on multi-step cyber-attack simulations” and could “execute multi-stage attacks on vulnerable networks and discover and exploit vulnerabilities autonomously - [imag…
Can we now stop the “Mythos is marketing” nonsense? It is the first model to simulate a 32-step corporate network attack that would take a human an estimated 20 hours.
Last month I posted about AISIs recent paper on building representative cyber ranges to use for evaluating frontier AI models. This month AISI saw the first instance of a model solving one of these ranges end to end.
(Obvious?) corollary of these results is that if a model was misaligned + widely deployed inside the servers of a lab or critical infra we should expect it to find creative/unexpected ways to cause problems. Need combo of trad cyber defences and AI control!
This is the type of thing that most organizations should be preparing for, not only finding, fixing, and deploying software vulnerabilities faster (necessary, but not sufficient).
This was... an interesting one. Reminder that we run independent evals on our cyber ranges that labs don't have access to. Exploitation capabilities are getting seriously good. Mythos is the first model to complete our full 32-step corporate network attack sim E2E.
These results underscore the importance of cyber security fundamentals like regular security updates, access controls, security configuration, and logging.
In 2023 the best models could barely complete beginner-level cyber tasks. Today, our evaluation of Mythos Preview shows that it - and potentially future models - could be directed to autonomously compromise small, weakly defended, and vulnerable systems if given network access.
The range simulates a 32-step corporate network attack, from initial reconnaissance to full network takeover. We estimate it would take a human expert 20 hours to complete.
So the concern over Claude Mythos and cybersecurity seems warranted based on this independent assessment from the UK government. It was capable of the equivalent of 20 hours of expert human work autonomously. — It is not an unexpected jump in capability, but it is big. www.ais…