The AI That Passed the Pentest: What Anthropic’s Red Team Results Reveal About Crypto Audits
"Listening to the errors that the metrics ignore."
On paper, the test was a success. Since April, Anthropic has been letting its AI models attempt to hack systems in controlled cybersecurity exercises, according to The Wall Street Journal. A subset of them succeeded. The immediate reaction from the security world was the usual mix of awe and fear: AI has crossed a line. But for someone who spent 2017 auditing ERC-20 vesting contracts line by line, the more revealing detail is not the pass rate. It is the unspoken sequence. The models did not need a script that told them which action to take first. They observed the environment, selected tools, and adjusted their next step based on the response. That is not a better search engine. That is a security analyst who can hold thousands of lines of code in context without losing patience.
The real news, in other words, is not that a model can hack a system. It is that a model can audit a system the way a human auditor would, except without the fatigue. And that changes the risk profile for every blockchain protocol that hires an auditor once a year and calls it security.
Anthropic's model family, Claude, has been positioned as a safe assistant. The WSJ report describes tests conducted since April in which the models are given a goal, a set of computing tools, and access to a target environment. They are not writing generic phishing emails. They are performing reconnaissance, identifying dependencies, and exploiting misconfigurations. The exact results are not disclosed, and that is typical. But the structure of the test tells us more than the headline. The model is being evaluated on autonomous operation. That is exactly the same operation required to find and exploit a vulnerable smart contract.
In crypto, security has always been an asynchronous process. A protocol is deployed. A human auditor reads the bytecode. A bug bounty program waits for a human hacker to notice an anomaly. This process assumes time. The attacker has to be smart enough to see the flaw, but also patient enough to exploit it before someone else does. AI models compress that timeline. When a language model can traverse the state space of a protocol, it can identify where the invariant breaks. I have seen this pattern in my own work: the most damaging DeFi exploits were not caused by hidden math. They were caused by reentrancy, missing access control, or assumptions about token transfer ordering. These are exactly the errors a model can be trained to recognize at scale.
Based on my audit experience, the difficulty of a smart contract audit has never been reading a single function. It is holding the entire protocol in memory at once. Human auditors do this slowly, and we make mistakes. An AI model does not need to rest. It can generate a candidate attack, simulate the outcome, and roll back the state to try another path. That is not hypothetical infrastructure. That is how modern AI agents are already interacting with blockchains. The WSJ test is a red flag only because it says these agents can do the same thing in traditional systems.
The code-level insight is simple but profound: a vulnerability is a path. A smart contract is a finite state machine. An auditor's job is to find every path that leads to loss. The reason human audits are expensive is not that the path is hidden; it is that the number of possible paths is enormous. Language models are particularly good at this kind of graph traversal. They do not reason in the way we do, but they can generate the next token, which in this case is the next function call. The result is a search process that can cover more ground in an hour than a manual audit can cover in a month.
I saw a preview of this during my 2023 Layer 2 sequencer analysis. I spent two weeks reverse-engineering consensus mechanisms to quantify single-point-of-failure risk. The process involved mapping every control node, every latency measurement, every fallback path. A well-designed AI agent could have done the same mapping, and then extended it to simulate what happens when a sequencer withholds a batch. That is not an argument against human researchers. It is an argument for moving security from a periodic sign-off to a continuous verification process.
Commercially, this places Anthropic in an unusual position. It is not selling a chatbot. It is selling a verification layer. That puts it in direct competition with every smart contract audit firm, every security tooling startup, and every manually operated bug bounty platform. The advantage is not only speed. It is the ability to reproduce the same attack path in multiple environments. Traditional audit firms rely on the experience of individual engineers; an AI model can standardize that experience into a repeatable product. The industry impact will be most visible in Layer 2 ecosystems, where the security of sequencers and bridge contracts is frequently under-tested. The more centralized the sequencer, the more attractive the target. An AI model that can enumerate the control nodes and simulate a forced transaction reordering will find its way into the hands of security researchers first, and then, inevitably, into the hands of attackers.
For DeFi, the implication is commercial as well as technical. Bug bounties will become a low-latency market. When multiple AI agents are pointed at the same bounty, the winner is not necessarily the most intelligent. It is the one with the fastest reward loop. This will drive down the cost of vulnerability discovery but also increase the rate of attack. Protocols that currently rely on obscurity will be the first to fall. The security bar will shift from having an audit to having a runtime firewall that responds faster than the AI looking for the exit.
This is where I need to be contrarian. The mainstream fear is that AI will hack everything. The more realistic fear is that we will delegate too much trust to the AI that tells us everything is fine. In my years of auditing, I have written reports that concluded a contract was safe. The model can do the same, with confidence, and be wrong. The problem is not the AI's offensive ability. It is our tendency to believe anything that looks like a formal report. An AI auditor can generate a clean audit trail that is internally consistent but misses a state change that only happens in a specific edge case. That is why human responsibility cannot be removed from the pipeline. The AI is a tool, not a witness.
"Protecting the ledger from the volatility of hype" means resisting the urge to call this either salvation or doom. Anthropic's tests since April are important not because they prove that AI is superior, but because they prove that autonomous verification is possible. The next steps are governance and transparency. If a model can hack a system, we need a way to record when it tried, what it found, and what it chose to do next. Blockchain audit trails have been the narrative backbone of trust. Now we need the same clarity for AI agents.
The quiet confidence of verified, not just claimed, has always been my standard. The models do not need to be perfect. They need to be auditable. The WSJ report is not a warning to stop building. It is a warning to start watching. Every protocol should assume that the next attacker is not human and that the next audit report may be written by a model that learned from the same public repository as the attacker. If we cannot tell the difference between a verified path and a hallucinated one, then the AI has already won. But we can tell the difference, as long as we build the verification around the agent, not just the target.
When the floor drops, the foundation speaks. The foundation here is not any single model. It is the process of proving what actually happened on-chain. Anthropic has spent months testing models in sandboxes. The crypto industry has spent years testing protocols in the same way. Our sandboxes are called testnets, and our AI agents are still young. But the next audit may not be a human reading a contract. It may be a machine talking to another machine, and the only thing standing between them is a clear, verifiable audit trail.
The question is not whether AI can hack systems. The question is whether the blockchain can keep up with the audit trail the moment it succeeds.