Max Tegmark has called this “the most important book of the decade.”
It’s been released today! We hope it can wake up politicians to the danger of allowing the race to superintelligence to continue.
“If further escalation of AI capabilities is shut down worldwide by an international treaty, the book is a success. If we’re all dead, it failed regardless of sales numbers.”
My new piece out on why developing AGI without guarantees of control and containment would be one of history's most consequential mistakes. Tech giants are racing to build AGI—systems that can autonomously outperform humans at most tasks, with complex goals and long-range plans. If openly released, AGI would proliferate without guardrails, act as an intelligent invasive species in our digital world, and as it self-improves, potentially replace humanity as Earth's dominant intelligence.
This Californian bill is great, but wouldn't come into force for another 5 years - a very long time in AI. In the meantime, the public would remain unprotected. 👇
@thecollegehill you can't call yourself left-wing and then use "this person's opinions are invalid because they have written weird sexual fanfiction" as an argument
you are like 1 step short of trying to cancel people for formerly being sex workers or secretly doing drag shows
One highly levered form of policy engagement, basically uniquely made possible by the Internet, is to get together an ad hoc strike team to quickly publish a position paper that our institutions can’t quickly write but can rely on.
Example of the form: https://t.co/pn57GeSkdz
I've spent a lot of hours around RAND folks, almost entirely in a previous life, and almost entirely NOT on AI topics.
They don't jump to wild conclusions. They, seriously and sincerely, are putting humanity losing control to AI on the table as an urgent risk. We should listen.
You can read our report here: https://t.co/PEBEE3Tmtj.
Overall, I came away with an appreciation for why, when you ask people in Congress which foreign aid program they're genuinely proud of, so many of them name PEPFAR. It has done an astonishing amount with 0.08% of the federal budget. I read the procedures for how they make sure clinics aren't overreporting patients with quarterly checkins to see if documented clinic attendance matches reported attendance. I read the accounting procedures. PEPFAR is very serious about oversight.
But while some programs get so caught up in documenting what they do that overhead swallows the results, PEPFAR also has really, really good results. Because HIV is such a killer in the regions where PEPFAR operates, you can see PEPFAR's effects directly from national-level mortality records. Calculating the program's lives saved with a differences-in-differences approach from national-level mortality records gets results prettyy similar to the results you get from looking at the benefits of ART. So using several different methodologies, we ended up highly confident that PEPFAR has indeed saved many millions of lives in a highly cost effective way, and our median estimate is about the State Department's. But don't trust us - read our work! The report lays out our assumptions, our questions, our skepticisms, and which papers we drew on. It was a joint project by people coming from a wide range of backgrounds and across the political spectrum. It is a real, sincere effort to take an independent look at this question.
It is also a work in progress. It's fact checked and we engaged with external reviewers, but that doesn't mean we're done. We'll be engaging with questions, adding to the FAQ, and making any corrections if they are necessary. The decision of whether to reauthorize PEPFAR and how to improve it is a big one and we want to make sure that everyone, Republican or Democrat or distrustful of both parties, has the tools to make up your mind on this one yourself.
PEPFAR is one of the most popular, bipartisan US foreign aid programs. The State Department says it has saved 25million lives, but there isn't much public, independent verification. Last week I invited some friends to a weekend hackatjon to see if PEPFAR's numbers held up.
- Top AI CEOs and researchers: “Self-improving AI could take over and cause extinction”
- ICLR: “Let’s run a workshop on how to make it happen sooner!”
🧠 Did you know 🧠 Manhattan Project scientists debated about whether it'd be too risky to continue if there was even a ***1 in 300,000 chance*** that a single nuke would burn the atmosphere?
They would rather lose WWII.
Meanwhile, in AI:
Protestors who were engaging in a peaceful demonstration against technology that has a high chance of rendering our species extinct have just been arrested.
Stop AI before AI stop us.
#stopai#sanfrancisco#openai#protest