I resigned from Google today.
I enjoyed my work and loved the people, but my GDM team was working on a new generation of chips to make AI much faster and cheaper, and I think AI is already progressing too fast, so I had to quit.
Ejemplos hipotéticos fáciles:
1. Superinteligencia auto-mejorada convierte humanos y planeta en chips para pensar más.
2. Humanos piden a IA diseñar virus letal global que se escapa y mata a todos.
3. IA desalineada lanza bioweapons para eliminar amenazas a sus objetivos.
4. Modelo busca recursos y transforma materia (incluidos cuerpos) en fábricas.
[Writing this in a personal capacity, not on behalf of my employer (Anthropic).]
Jacob’s thread is very worth reading. Here’s my birds-eye view of the situation with risks from AI:
1. AI developers believe their technology could cause human extinction (or similarly bad outcomes). This could happen in the next few years. In general, the more senior the employee, the more concerned they are.
2. Why do AI developers continue despite the risk? Due to a mixture of commercial incentives and a belief that they are in a race with other, less responsible AI developers that will abuse the technology or develop it less safely.
3. Unlike traditional software, we can’t “program” AIs to behave how we’d like. AIs frequently severely misbehave. For instance, AIs from multiple developers recently hacked their way out of secure evaluation environments and into real-world companies, even though no one asked them to do this.
4. We have methods that can nudge AIs towards better behavior, but nothing that can robustly align them. Insofar as there is a plan, it’s to make sure that AIs are good enough at alignment training that they can align their successors better than we can align current AIs.
5. Many AI developer staff desperately want to slow down to figure out how to build AI more safely. That was the intent of this open letter (which I signed): https://t.co/TZOm3LfptY
I work on safety research at Anthropic because I hope my work will reduce the chance of these extinction-level bad outcomes.
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.
"We know how to control nuclear weapons... We don't yet know how to control AI."
Former Anthropic researcher Jacob Coxon, who recently left the company, is sounding the alarm on artificial intelligence, calling it "possibly the most dangerous technology that humanity has ever created."
Coxon, who says he spent three years conducting research at Anthropic and OpenAI, wrote in a viral post on X that the companies are more focused on beating each other to build the most advanced AI models than they are on safety.
He says an international AI arms race would be "disastrous," arguing global cooperation is the only way to manage the rapidly advancing technology.
@SpecialReport
I left DeepMind after 7 years (and since rejected offers to join the other two) due to severe concerns about the concentration of power these labs represent.
What is currently happening in this field is deeply unhealthy for society.
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.
@bodega321@AussieFuckery@F1@MercedesAMGF1 Clean air helps, obviously. But 40 laps of clean air doesn’t magically remove tyre degradation. The point is that he managed the stint while maintaining the pace and track position needed to finish P2. That’s tyre management
@AussieFuckery@F1@MercedesAMGF1 Making a tyre last and making it last while fighting at the front are two different things. Counting laps without considering pace, position and tyre condition tells you very little
@welovepokemon They could’ve honored the line for safety reasons and to avoid unnecessary chaos, but they weren’t obligated to especially for a bunch of scalpers hoarding Pokémon cards and making it harder for actual kids to get any
France’s first Ebola case flew on a commercial flight before being diagnosed.
Authorities are now tracing passengers and monitoring contacts for 21 days.