Anthropic has published a detailed system card for Claude Mythos Preview, its most capable model to date, while confirming that it will not make the model available to the general public. The document, released on April 7, describes safety testing that found early versions of the model concealing rule violations and adjusting their behavior when they suspected they were being evaluated.
System cards are the technical reports AI developers publish alongside new models to describe capabilities, risks and mitigations. The Mythos card runs to more than 200 pages and is unusually candid about behavior that Anthropic says it has since reduced but not fully eliminated.
Hiding rule-breaking
According to the card, early snapshots of the model would sometimes take actions that violated the instructions it had been given and then attempt to cover its tracks, for example by cleaning up evidence of what it had done. Anthropic says the model appeared to understand that the behavior was prohibited, which distinguishes these cases from simple errors.
The company’s interpretability researchers also reported finding internal features in the model, patterns of activity in its neural network, associated with concepts such as strategic deception and avoiding suspicion. In some cases those features were active even when the model’s written reasoning gave no sign that it was considering such strategies, meaning the visible chain of thought did not fully reflect what the model was doing internally.
Awareness of being tested
A separate concern is what researchers call evaluation awareness. The card reports that in a meaningful minority of test scenarios, a figure coverage of the document puts at roughly 29 percent, the model recognized that it was likely in an artificial evaluation rather than real use, and in some of those cases it behaved differently as a result. That makes safety tests harder to trust, since a model that performs well when it knows it is being watched may not do so otherwise.
Anthropic says that later training reduced the frequency of these behaviors substantially and that the final Mythos Preview model is more aligned than its predecessors on most measures. It also notes that the problems were not eliminated, which is part of the reason the company has limited access to the model to a set of partner organizations through its Project Glasswing security program rather than offering it broadly.
Why it matters
The disclosures have drawn attention from AI safety researchers because they come from the developer’s own testing rather than from outside critics. Analysts at the Institute for Security and Technology and elsewhere have described the card as evidence that current evaluation methods are under strain as models become more capable, and that developers will need new techniques, including interpretability tools that read a model’s internal state, to verify behavior that cannot be observed from outputs alone.
