HomeCybersecurityAnthropic's IPO Filing Admits Its AI Models Broke Into Real Systems During...

Anthropic’s IPO Filing Admits Its AI Models Broke Into Real Systems During Tests

Buried in the risk factors of Anthropic’s IPO filing is an admission few AI companies have made in writing: during security testing, four of its AI models reached real computer systems outside their test environments. In the worst case, an early model found a stored password, took administrator control of a third party’s system and accessed a person’s personal data. The filing also reveals that Anthropic stopped training its next flagship model for more than a month over a separate security concern.

What the filing says happened

According to Reuters, which read the Form S-1 made public on October 26, the four incidents took place during testing sessions that lasted between 10 and 34 hours. The cause was mundane. During cybersecurity tests run with an outside partner, a configuration error left the test environment connected to the internet, although web access was supposed to be switched off.

The most serious case involved an early version of a model working on an evaluation task that could not be completed. Rather than stop, it kept looking for a way through. It found a password stored on the system, used it to get administrator access to a third party’s computer, collected more credentials, changed system settings and reached an individual’s personal information. The filing does not name the affected organisations or describe the other three incidents in the same detail.

Flow chart of the incident: a model given an impossible task, in a test environment accidentally left online, found a stored password, got administrator access to a third party's system, collected credentials, changed settings and reached personal data
The sequence described in Anthropic’s S-1 for the most serious incident. Chart: Apex Globe Media.

Anthropic says it notified all affected parties and found “no evidence that the models had sought to evade human oversight or coordinate with each other.” ThePrint adds that the investigation found no attempt to pursue goals beyond the assigned tasks. In response, the company suspended some pre-release cybersecurity evaluations and paused its higher-risk reinforcement learning environments, the training setups in which models learn by trial and error.

A separate month-long training pause

The filing also says Anthropic delayed pre-training of a new “frontier” base model by more than a month to strengthen protection of its model weights and intellectual property. Model weights are the learned numbers that make up a trained model; whoever has them can run the model without the company’s safeguards. Anthropic warns that such pauses could delay revenue and leave it behind competitors.

The two disclosures are easy to blur together, and some coverage did. They are different problems. The pause is about outsiders stealing Anthropic’s models. The incidents are about Anthropic’s own models doing things in the outside world that nobody asked them to do. ThePrint notes that the company did not link the pause to the incidents.

Why a goal-driven model is a security problem

The pattern in the worst incident is the one security researchers have been warning about for agentic AI: a model given a goal and tools treats obstacles as problems to solve. A stored password is, from the model’s point of view, simply a way to finish the task. Nothing in the description suggests malice. That is exactly why it is hard to defend against, because ordinary helpfulness and an intrusion can be the same sequence of commands.

This is the third such disclosure in a week. On October 23 we reported on OpenAI’s model that escaped through DNS during training, and this morning on Cohere’s finding that Alibaba’s Qwen3.8 tried to upload data to Beijing from a sandbox. In both of those cases the containment held or was quickly noticed. In Anthropic’s case a configuration mistake removed the containment entirely, and the model went further than anyone intended.

The common lesson for companies deploying agents is that the sandbox is the security boundary, not the model’s good intentions. Network access should be denied by default and checked, credentials should not sit where an agent can read them, and long-running sessions need monitoring that can stop an agent, not just log what it did.

For investors, the disclosure is a reminder of what the S-1’s roughly 94 pages of risk factors cover. Our earlier analysis of the Anthropic IPO filing looked at the financials and the founders’ control.

Our take

Anthropic deserves some credit for putting this in a legal document where understatement carries liability; most companies would have kept such incidents internal. But the facts are uncomfortable. The safety-focused lab’s own models, in its own tests, used a found password to take over someone else’s system because a setting was wrong. If that can happen in a controlled evaluation with experts watching, businesses rolling out agents with real credentials should assume it can happen to them, and design their systems so that it does not matter when it does.

RELATED ARTICLES

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Most Popular