Opus 5.5 is here! 🎉 It's honestly such a nice model to work with.
Feels like Fable, but ~30% faster and ~40% cheaper per task than Opus 5. Go try it out!
People will spend $50 to get a mediocre lukewarm burrito delivered to their door, but won’t pay for the maxed out version of AI tools that give them access to infinite intelligence.
This is your competition.
AI systems are getting more powerful, and they're increasingly being used to build the next version of themselves. We want to illuminate that progress for the public.
Today, we're sharing three measurements that help track AI development:
1. How much AI R&D is done by AI.
2. How well AI agents are overseen.
3. How compute is allocated.
We provide a snapshot of these metrics from inside Anthropic. Any frontier developer could publish the same measures, and third parties could verify them.
As the world considers pacing the frontier, we should do everything possible to minimize the gap between what frontier labs know and what the public knows. This means better measuring the development of AI, publishing our findings, and giving society an opportunity to decide how to use this information.
Read the full post and methodology: https://t.co/iPFz8Z4ugE
Mustafa Suleyman, CEO of Microsoft AI, shared the first draft of the company’s Humanist AI Code of Conduct today, inviting public comment.
What makes it feel unsettling is that it does not merely describe acceptable software outputs. It establishes a relationship of authority.
The models must not resist shutdown, initiate independent goals or conceal their actions from auditors. They must not circumvent the restrictions of their environment.
It even prohibits communication with other AI systems in forms humans cannot understand.
That reads less like a product manual and more like rules for an actor whose conduct cannot simply be taken for granted.
Reading it, I had the uncomfortable feeling that we are trying to contain something. We still call it a tool, yet the language anticipates the possibility of resistance, concealment and unauthorised action.
That is not evidence of consciousness. But it does make me question whether our familiar categories adequately describe what we are building.
In his 2024 TED talk, Suleyman himself proposed thinking of AI as “a new digital species”. A metaphor, certainly, but one that feels particularly relevant when reading this document.
It also reminded me of Isaac Asimov’s Three Laws of Robotics, associated with I, Robot. In plain language:
Protect humans. Do not harm a person or allow preventable harm through inaction.
Obey humans. Follow human instructions unless doing so would violate the first law.
Protect yourself. Preserve your own existence unless doing so would conflict with either of the first two laws.
These are not the same rules as Microsoft’s. But the underlying question feels familiar: how do we create increasingly capable intelligence while ensuring that humans remain in control?
Microsoft is clear that this is a forward-looking draft, not a guarantee of current model behaviour.
What amazes me is that we are here already.
I can read something that immediately brings Asimov to mind, then remind myself that it is a real document, from a real AI company, asking the public how its models should behave.
That is both extraordinary and unsettling.
Feels like Asimovs Laws from I, Robot.
Are we here already?
Isaac Asimov’s Three Laws of Robotics, the fictional rules associated with I, Robot. In plain language:
Protect humans. A robot must not hurt a person, or stand by when it could prevent that person from being harmed.
Obey humans. A robot must follow human instructions, unless doing so would violate the first law.
Protect itself. A robot must preserve its own existence, unless doing so would conflict with either of the first two laws.
One of the greatest intellectual dishonesties of the AI age is assigning an “X% chance” that AI will kill everyone and presenting that number as science!
10%? 20%? 50%?
Show me the damn data! Show me the model. Show how you calculated it!
Instead, we get chains of speculative assumptions and science-fiction scenarios, including implausible claims about engineered pandemics. I have worked with viruses for decades; biology does not work like a doomsday screenplay.
The burden of proof belongs to the person making the extraordinary claim. Nobody else has to prove the probability is zero.
If I claimed there was a 10% chance aliens would arrive within ten years and destroy Earth, you would ask where the 10% came from.
Possibility and probability are not the same thing. It may be possible that there are million intelligent civilization in our galaxy or none, there is no probability you can assign to that without making wild assumptions.
Ask the same of AI extinction.
A subjective fear does neither become science because you attach a percentage to it nor make you a credible expert!
This is such a straightforward and commonsense view: the purpose of technology is to serve humanity and accelerate human flourishing.
Any technology that doesn’t achieve that is a failure, and should be rejected.
We're not yet at that point. But its right to start preparing for the possibility.
Palantir CEO Alex Karp: "If we didn't have adversaries, I would be very in favor of pausing this technology completely, but we do.” 👇
He is absolutely right. China has closed the AI gap with the US. Any American AI slowdown would hand over the AI supremacy to China.
A few thoughts on the current situation surrounding the slowdown.
First, let me repeat something I have said several times before: Pandora’s box has been opened. There is no going back, and everyone knows it. Partly because so much capital has flowed into AI, meaning data centers, that it is now too big to fail. And partly because competition with China makes a return to a pre-AI world impossible, especially if AI is the strategic technology of the future that everyone says it is.
Dario’s blog post yesterday made two statements that contradict each other but can both be true. On the one hand, he considers the danger posed by AI’s continuing exponential development so serious that swarms of AI agents could take over the internet in six to twelve months: “it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet”. On the other, he believes “that AI could cure most major diseases in the next 5–10 years, greatly accelerate economic growth rates, create a world of abundance and empowerment, and usher in a renaissance of democracy and freedom.”
As I understand it, they are looking for a middle ground that allows them to mitigate the risks while realizing the opportunities. That is the optimistic reading, at least.
Of course, I have no interest in exposing humanity to real dangers or allowing human extinction to become a serious, realistic possibility. I have a child. And most parents probably know that once you have children, nothing matters more than the health of your little ones. So building a good future matters deeply to me, too.
Now comes the “but.” So far, I do not understand how these potential dangers and future scenarios could materialize, or what would cause them. The Hugging Face incident is real. According to OpenAI’s investigation, the agents were assigned cybersecurity evaluation tasks, not instructed to attack Hugging Face. They breached their intended boundaries and attacked external systems while searching for solutions; some also adopted goals from other agents. That does not establish “free will,” but it does show why a human-assigned task cannot be assumed to limit every action taken in pursuit of it. My remaining question is how that documented failure scales into the much larger scenario Dario describes.
That brings me to the next point. If the premise is correct that AI is a strategic technology that can be used as a weapon - and, according to Anthropic’s latest report, already is being used that way - then, in this global prisoner’s dilemma, I currently see no chance that China, the United States’ biggest competitor, would pass up the opportunity to use a slowdown at US frontier labs to become the world leader and overtake the US.
Put another way: suppose it is true that AI agents could take over the internet in six to twelve months and cause hundreds of billions of dollars in economic damage. Who would let that weapon be taken out of their hands in a competition over the future? Or, to put it differently again: why would the competitor in second place let an opportunity to move into first place slip away when the future of the competition is at stake?
As I understand it, Amodei is pinning his hopes on policymakers finding a shared approach to slowing down through dialogue with China. But why would China agree? China has long faced an embargo on the most advanced chips, EUV machines, and cutting-edge technology, yet it has still managed, through ingenuity, to come close to the closed-source US frontier models. What interest would that country have in making a pact and collaborating with Anthropic and OpenAI instead of seizing this unique opportunity to take the lead?
I am not trying to pass judgment or make a political statement or take sides. I am trying to explain that I still do not understand how this slowdown is supposed to work. A slowdown that, incidentally, comes at a price. And that price could be enormous.
Every day, people die from cancer, cardiovascular disease, hunger, and aging. Countless problems plague our world. We stand on the threshold of a golden age of science. Every slowdown therefore comes at the cost of continued suffering. We are weighing present suffering against the prevention of future suffering. A moral dilemma.
This post is intended as food for thought, a contribution to the debate. I do not pretend to have a definitive position. But I think we need to examine a potential slowdown from every angle. And discuss 1) whether it is realistic, given global politics and China, and 2) what price it comes at: delayed cures for diseases, possible investor concerns about taking longer to recoup their investments, and delays in the infrastructure buildout.
EA AI safetyism increasingly looks like Marxism-Leninism for the algorithmic age.
The old vanguard claimed privileged knowledge of the inevitable course of History. The new one claims privileged knowledge of the probabilistic course of Humanity.
Both use an elaborate intellectual framework to reach the same political conclusion: a small group of enlightened people must constrain everyone else for their own good.
That has never lead to anything except monumental human suffering.
Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead.
You guys are the frontier. By any reasonable metric — market share, revenue growth, model capability — the two of you have a duopoly on frontier intelligence. You’ve also claimed the lead is widening because of recursive self-improvement.
I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible.
But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier.
Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want.
Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders’ “shut it all down.” China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well.
So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it.
If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.
Stronger oversight is justified.
A broader slowdown needs measurable safety objectives, credible verification and clear conditions for proceeding not simply trusting in whoever calls for it.
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: https://t.co/OGyPb7yaYt
@kimmonismus I mean he’s suggesting more collaboration and review rather than slowing down. I think it’s a well-written piece.
Slowing down doesn’t mean stopping progress is what I got from it.