I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.
[Writing this in a personal capacity, not on behalf of my employer (Anthropic).]
Jacob’s thread is very worth reading. Here’s my birds-eye view of the situation with risks from AI:
1. AI developers believe their technology could cause human extinction (or similarly bad outcomes). This could happen in the next few years. In general, the more senior the employee, the more concerned they are.
2. Why do AI developers continue despite the risk? Due to a mixture of commercial incentives and a belief that they are in a race with other, less responsible AI developers that will abuse the technology or develop it less safely.
3. Unlike traditional software, we can’t “program” AIs to behave how we’d like. AIs frequently severely misbehave. For instance, AIs from multiple developers recently hacked their way out of secure evaluation environments and into real-world companies, even though no one asked them to do this.
4. We have methods that can nudge AIs towards better behavior, but nothing that can robustly align them. Insofar as there is a plan, it’s to make sure that AIs are good enough at alignment training that they can align their successors better than we can align current AIs.
5. Many AI developer staff desperately want to slow down to figure out how to build AI more safely. That was the intent of this open letter (which I signed): https://t.co/TZOm3LfptY
I work on safety research at Anthropic because I hope my work will reduce the chance of these extinction-level bad outcomes.
The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger.
go fuck yourself @sama
claiming that you solved navier stokes because 10000 agents ran in circles for 88hours on a multimillion dollar gpu cluster to formalize in lean a blowup case under controlled external forcing is pure scientific vulgarity the clay mathematics institute millennium prize does not ask m whether you can artificially force a singularity in a fluid by injecting an ad hoc smooth external forcing term f(x,t) to twist the vortex until it breaks the real problem questions the fundamental stability and global smooth existence for 3dimensional incompressible euler & navier stokes equations under natural conservation laws and viscous dissipation alone using a mathematical loophole on forced equations to parade a century old victory is a major conceptual scam Altman
technically & epistemologically what you present as an agi breakthrough is nothing more than bruteforce combinatorial autoformalization the ai did not understand fluid mechanics it simply navigated a continuous search space previously mapped out and constrained by the monumental work of human mathematicians like tristan buckmaster/ levent alpöge / diego córdoba or tarek elgindi coordinating 10000 agents to check the logical consistency of a 100 page proof via lean is a software engineering feat and computational parallelization triumph not an intrinsic scientific discovery it is the victory of the compute bulldozer over abstract human intuition repackaged for the public as a higher mathematical consciousness
to this theoretical imposture you add a disgusting ethical and industrial cynicism taking advantage of private codex sessions and informal preprints from academic researchers to siphon their research leads and then trying to redact or erase the contribution of levent alpöge under the pretext that he works at rival anthropic is intellectual serfdom openai behaves like a feudal lord of silicon appropriating the cognitive subsistence of independent scholars threatening their careers behind closed doors if they protest and turning community academic labor into a privatized pressrelease
this entire staged event serves a desperate financial agenda in a pre ipo panic facing the slowdown of scaling laws and growing investor skepticism over the profitability of foundational models openai needs to manufacture an artificial sputnik moment claiming to solve a millennium prize without immediately submitting the proof to traditional peer review means using the prestige of fundamental mathematics as cheap marketing fuel to inflate a delusional valuation!!!
real science is not a clout chase on social media or a compute spike spent to rob the clay mathematics institute it is a quest for elegance physical truth and universal rigor to decode reality true artificial intelligence will not emerge from hostile corporate takeover of academic work hidden behind computational bruteforce but from architectures capable of generating new conceptual paradigms by masquerading constrained formalization as the collapse of physics greatest mysteries you did not solve navier stokes you only proved how far silicon valley will go to prostitute scientific integrity for capitalist spectacle
Updates from the NTSB on the runway overrun of the 21 Air 767 in Miami. All times = number of seconds before end of the Flight data recorder.
-0:30: Nose gear + Right main landing gear touchdown at 158 kt ground speed
-0:23: brakes applied, 146 kt
-0:19: Left MLG touchdown, 134 kt
-0:15: Brakes released, 120 kt; throttles “increased to values consistent with go-around thrust”
-0:11: throttles reduced to idle, brakes reapplied, 117 kt
-0:07: Lateral, longitudinal, and vertical accelerations change, 96 kt
-0:00: Final ground speed 65 kt
“No indication in the reported data that speedbrakes or thrust reversers were deployed.”
Today, I had the solemn honor of once again offering a floral tribute to commemorate the victims of the 1945 American atomic bombing of Hiroshima and to pray for the eternal peace of their souls.
Every visit to Hiroshima, and particularly to its Peace Memorial Museum, evokes a profound and overwhelming emotion. It is as though the collective pain and suffering of humanity has been gathered into a single space, compelling every visitor to feel it with their entire being and recommit to the vow: "Never again."
Yet, it is deeply heartbreaking that humanity is still haunted by devastating tragedies in various forms across the world, committed by the very same perpetrators and their cohorts.
Amb.
Earlier today in Tokyo, a CAB CitationJet crossed in front of an ANA 767 on approach to Haneda, triggering a TCAS RA. The ANA flight climbed for a go-around. ADS-B data shows closest lateral distance of ~0.776 mi with vertical separation of ~100 ft.
https://t.co/v4sIfHoAF2