[Writing this in a personal capacity, not on behalf of my employer (Anthropic).]
Jacob’s thread is very worth reading. Here’s my birds-eye view of the situation with risks from AI:
1. AI developers believe their technology could cause human extinction (or similarly bad outcomes). This could happen in the next few years. In general, the more senior the employee, the more concerned they are.
2. Why do AI developers continue despite the risk? Due to a mixture of commercial incentives and a belief that they are in a race with other, less responsible AI developers that will abuse the technology or develop it less safely.
3. Unlike traditional software, we can’t “program” AIs to behave how we’d like. AIs frequently severely misbehave. For instance, AIs from multiple developers recently hacked their way out of secure evaluation environments and into real-world companies, even though no one asked them to do this.
4. We have methods that can nudge AIs towards better behavior, but nothing that can robustly align them. Insofar as there is a plan, it’s to make sure that AIs are good enough at alignment training that they can align their successors better than we can align current AIs.
5. Many AI developer staff desperately want to slow down to figure out how to build AI more safely. That was the intent of this open letter (which I signed): https://t.co/TZOm3LfptY
I work on safety research at Anthropic because I hope my work will reduce the chance of these extinction-level bad outcomes.
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.
"The historic apology, delivered in parliament, comes after a report found that 200,000 children and vulnerable adults had suffered abuse while in state and faith-based care between 1950 and 2019."
https://t.co/2ZDLu8ucII
The Freedom From Religion Foundation is sounding the alarm on an underreported threat to democracy posed by Project 2025: its intention to defund and muzzle public broadcasting. https://t.co/Gq9SIgRPxq
"A Church of England bishop has called on the Archbishop of Canterbury to resign, calling his position "untenable" after a damning report into a prolific child abuser associated with the Church."
https://t.co/JggZLVvVbl
In the last interview of his life (1996), astronomer Carl Sagan warned of the dangers that come when citizens cannot ask skeptical scientific questions of those in authority. Watch and ask: was he right?
As usual, this is not true.
The Liberals did not ban prayer. The Chaplain General issues directives independently intended to be more inclusive, but the Conservatives can’t rage-bait with the truth.
@emayeg2 @damccaig Look at me go. Set one up. Don't know how active I'll be, I have a threads accnt as well which I'm not using yet. But say "Hi" on Bsky and maybe I'll get rolling!
@according2luke.bsky.social
And thanks again for thinking of me!
This level of incompetence is dangerous.
@Alberta_UCP & @ABDanielleSmith have made the unprecedented, drastic decision to fire the entire AIMCo executive & take partisan control of the $170b fund.
This creates uncertainty in the management of the pensions of 375k Albertans.
The World's richest man who has been speaking to Putin regularly for the last couple of years joined Trump on his call with Zelensky.
It's all completely bonkers...
https://t.co/vpYT19Ddxa