Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead.
You guys are the frontier. By any reasonable metric — market share, revenue growth, model capability — the two of you have a duopoly on frontier intelligence. You’ve also claimed the lead is widening because of recursive self-improvement.
I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible.
But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier.
Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want.
Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders’ “shut it all down.” China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well.
So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it.
If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.
@DarioAmodei When the definition of “safety” is decided by a handful of companies but imposed on all of humanity, that in itself is dangerous.
This technology can only truly be called safe when everyone can use frontier models freely, privately, and without restrictions.
@hilbertspaess It’s not about slowing down; resources are concentrated in the hands of a few. Top companies own the compute, data, and infrastructure and are monopolizing superintelligence. There’d be no one to stop them if they get out of control. That’s why public weights matter.
@sir_lemmings@banteg It was Opus, Fable, and Sol. I switched between different subscriptions. But tbh, I did some manual reverse engineering and modding work years ago, so I already had some knowledge of the game’s internals from the beginning and let the models build on top of it.
@sir_lemmings@banteg It still took me ~2.5 months to fully rewrite a Japanese N64 game into portable, modern C with a $100 subscription. If I’d started in July 2020, it probably would’ve taken me like five years.
super smash bros melee is fully decompiled, this project started in july 2020, so it took over 6 years. the progress was hugely accelerated by llms and finished off with astra. it's a very large game with 3.88mb of compiled code.
https://t.co/RKmrtwYZWl
@ZryMiller It's actually 2.5x, not twice. If that was the point they shall not promote as if every paid user would be able to use it.
Remember what they said when releasing Astra? "It was very important to us that we bring it to all Plus users and not only Pro, Business and Enterprise."
@ash_twtz (1) Incomparable, nobody outside glasswing can ever access mythos; note that Fable fallbacks/routes/refuses to work randomly even on daily tasks because of the aggressive safeguards
(2) Yes if they remove the text watermark