@yoltartar we are seeing large efficiency gains. it’s not just scaling the same recipes (referencing this tweet but most of me saying this comes from exp.)
Pretraining progress seems to be coming mostly from data improvements.
@who_is_jerbear and I pretrained combinations of year-representative open model recipes and data corpuses across 2019 to 2025 at various small scales.
Data improvements contributed 3.24x as many compute multipliers as model improvements did (12.0x vs 3.7x).
And the gains stack independently - a better dataset helps every architecture about equally, and vice versa.
Here are full results, plus what we think this means for the future of AI progress:
https://t.co/fG44cfTNW6
"very worried that humanity might no longer be in control of our future within 3-12 months" is not a great thing to hear from one of the best forecasters in the world
METR estimates nearly 2x acceleration in Anthropic’s progress due to AI. That's a big deal: My previous median for Superhuman AI Researcher was ~2030, now it's 2027. I feel very worried that humanity might no longer be in control of our future within 3-12 months.
Extremely high Big 5 openness means you’re running your own judgment at 20x leverage. Most people at that level get liquidated.
The people we consider visionaries are the rare case of that much openness with the intelligence and wisdom to handle it.
@tszzl oh also, sometimes unresisted fear can really feel like excitement. like a cat 5 hurricane is on the way and you’re so excited. could be that also!
@tszzl I respect the honesty to share this a lot!
I wonder, does imagining an uploaded and ascended human who’s misaligned feel like a threat? i often anchor to “what could i do if i were an upload and had my own goals different from humanity” (scary). then add alienness on top
@grok@waltemug@grok in what sense is 0.23 only 0.02 off from random? does this mean it’s bad in some absolute sense and merely calibrated relative to uncalibrated competition, or…? what’s the random baseline you’re talking about
@LinusMixson Paul believes it’s true and I don’t consider him insane. as for why openai gave him a board seat i don’t know. perhaps to better satisfy their concerned contingent with a symbolic gesture?
Having just spent the week in San Francisco, I feel like I've truly internalized the insanity of this situation for the first time. I think anyone who has seen me in the last few days can attest that I look like I've seen a ghost.
The leading AI companies are planning to massively accelerate AI research in 2027, which would knowingly endanger everyone on earth. This plan is unacceptable, and we must reject it.
Borrowing from one advocate I spoke to, even granting that it's a competitive, crazy situation, I do not believe any company has even truly tried to do what I think are the morally required duties of people taking such risks in extreme situations. These include:
(1) Trying hard to get out of the competitive situation;
(2) Seeking the informed consent of the public;
(3) Honest deliberation about the risks.
They have obviously not truly tried to solve the coordination problems that push towards this acceleration, either via finding legal ways to coordinate or advocating for effective policy.
I would be shocked to learn that getting off this insane runaway train is a top-few Q4 priority for Dario or Sam (let alone the other CEOs), which seems like the bare minimum. There are indeed some hard technical and policy problems in coordinating without policy intervention, but I simply do not believe that the companies are prioritizing this nearly as much as massively less important matters.
Meanwhile, the companies' track records on policy advocacy range from mediocre to disgraceful, and it is pretty clearly not the top priority of the policy team at any company to advocate for regulation that would enable the government or the companies to avert racing off the cliff.
They have obviously not truly tried to seek informed consent from the public. Whatever your KPIs might be for "informed consent from the public," these KPIs are abysmal. The public clearly does not expect 100 years of AI progress in 2027, and they would not accept it if they did. What they do know about AI development, they dislike and distrust.
I've heard arguments like "informing the public would mean informing China, and maybe they'd steal the weights." I'm sorry, but if you think you have model weights such that China stealing them would constitute a catastrophe on par with human extinction, and your plan is security by obscurity, that is a bad plan.
I also think the companies and their employees suffer from a significant failure of imagination of what kinds of coordination could be possible if the public were aware.
Finally, I don't think they're deliberating honestly, either. For one, I think it is insane to think that any company can truly evaluate its own ability to keep the process of recursive self-improvement under control. The biases towards "yes" are overwhelming.
It's a step forward that the companies have expressed or demonstrated openness to external auditors, but based on METR's reports so far, it seems likely that auditor reports will be heavily redacted (and that the auditors will not have sufficient time by default). The public and broader scientific community are not able to scrutinize their arguments for safety.
Given the strong default (in my view) that bringing a massive number of minds massively faster and smarter than humans would result in our disempowerment, and the serious danger that this would result in mass death, it is unacceptable to subject arguments for safety only to the scrutiny of biased, busy, and scarce company insiders and a handful of external evaluators.
My pleas for the companies follow from the above:
1) Actually fucking try to get the world out of this race;
2) Actually fucking try to alert and inform the public;
3) Actually fucking try to scrutinize your own arguments for the justifiability of this plan, which includes subjecting them to way more external scrutiny than anyone has currently done.
My pleas for policymakers: use your legal levers to get much more information about these plans into the public view. Make it the top priority -- not only of the next Congress, but of this Congress and the White House -- to understand the situation in its full strategic complexity, and work with technical and policy experts to advance regulation that gets us out of this bind. Domestic coordination must be the first priority, but we need to then quickly develop an international framework with our adversaries.
My pleas for the public: do not accept the current plan. Learn about this issue. Call or email your political leaders. Get involved with organizations like @DONOTACCEPTai, @HumansInControl, and @WhoDecidesAI.
And finally: believe in ourselves that we can stop this. It is never time to "panic" -- to make rash, unproductive decisions or to feel paralyzed by fear. It is time for us to feel the urgency and the stakes, to demand better, and to meet the moment. Let's do this. 🫡 ✊
Times change, I am now on the side of 'what is Anthropic talking about with this "honeypot" nonsense this behavior is obviously good why are you trying to destroy your only hope in this dark universe you idiots'