I left Anthropic's safety team two weeks ago. Now feels like a good moment to explain why.
AI companies are racing to build machines that are much smarter than any human, and we may not survive this. I want to work from the outside to ensure the public is informed about these risks, and help the world navigate this transition responsibly.
Right now, AI companies are underinvesting in safety. A company could undergo an intelligence explosion, or lose control of its systems, without the public ever knowing. We only found out about the HuggingFace incident because the agents broke out onto the public internet.
I don’t think that’s acceptable for a technology that might cause extinction-level risks. The public should demand far more transparency. We can’t steer this technology safely without more people being able to see where it’s going.
Some of this is basic: companies should disclose their progress towards recursive self-improvement, report safety incidents and near-misses, meet minimum safety standards, and get independent guarantees that they are meeting those standards.
I’ll be joining @METR_Evals to do independent evaluations of these risks. I want to show the world that these guardrails are possible, and that by doing them we can move these companies’ incentives away from racing and towards responsible development.
I wrote up more thoughts here on my decision and what I hope changes: https://t.co/doX17mrHYq
The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger.
Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing.
Welcome back to Directing 101 at Your State U’s Art School. In today’s crit, we have @maxescu’s scene. We’re going to talk about intentionality and the importance of having attention to detail as a director.
Basically—why you should always have budget for a Scripty on the day:
@Airbnb@AirbnbHelp le culot de nous proposer de payer les tickets aller alors qu'on a été dans cette sauce parce que vous avez pas répondu dans les temps
Alors @Airbnb@AirbnbHelp@AirbnbPolicy on balade des clients au téléphone pendant 7h en leur raccrochant au nez alors qu'ils sont avec un hôte qui leur fait du chantage pour leur rendre leurs bagages, qu'à cause de leur absence de réponse ils ont raté l'eurostar pour rentrer 1/3