MIRI

The Machine Intelligence Research Institute exists to maximize the probability that the creation of smarter-than-human intelligence has a positive impact.

[MIRI @MIRIBerkeley]

#7 Combined Print & E-Book Nonfiction
#8 Hardcover Nonfiction

[The Spectator @spectator]

'Some people believe that if machines decide to kill us it's the right thing to do because they're smart'

AI researcher Nate Soares reveals that some factions in Silicon Valley mistakenly believe that if an AI is exceptionally intelligent it must also be highly moral. @So8res @Freddygray31

[MIRI @MIRIBerkeley]

This just passed 2M views! If you haven't seen it yet, check out this AI In Context video on "If Anyone Builds It, Everyone Dies"

https://t.co/y0luTYbtL0

[AI StopWatch @AIStopWatch]

Ten places where Magnifica Humanitas matters for AI.

At 42k words long, Pope Leo XIV’s new encyclical has a lot to say. In our most recent Digest, Mitchell Howe outlines the parts which might be the most impactful.

[AI StopWatch @AIStopWatch]

What will be the impact of AI industry super PACs?

"The takeaway here is that this year’s U.S. midterm elections are being aggressively shaped by different factions of the AI industry sometimes supporting the same candidates, sometimes different candidates, buying ads that don’t have anything to do with AI."

[MIRI @MIRIBerkeley]

If you're interested in creating videos about the extinction threat posed by superhuman AI, consider applying to this bootcamp!

[Aella @Aella_Girl]

First round of applications for PDKU close in two days! We'll send out acceptance letters by June 1st.
You should apply now! It's gonna be fun and you'll make friends and do weird shit
(U can still apply after that, but there will be fewer slots and your chances will be lower)

[Elizabeth Barnes @BethMayBarnes]

Our report focuses on claims that are (1) solidly defensible and (2) generally agreed within METR. Here I’ll give some personal opinions on how we should feel about the state of AI risk, and the IMO most important limitations of the report.

[AI StopWatch @AIStopWatch]

"Gathering information is perhaps an important step forward, but it's not nearly enough."

In today's Digest, Joe Rogero discusses the new executive order from CA governor Gavin Newsom.

[MIRI @MIRIBerkeley]

https://t.co/o06QEIKjSM

[MIRI @MIRIBerkeley]

An internal model at OpenAI has autonomously disproved a central conjecture in discrete geometry, a mathematical field with applications in cryptography, wireless device communication, and medical imaging. The proof relates to a famous question posed by Paul Erdős in 1946. It has been verified by prominent mathematicians in a companion paper.

The verifying mathematicians consider this to be a genuinely novel breakthrough on one of the most discussed problems in this area of mathematics. One called it “arguably the best known problem in Discrete Geometry.” Another observed, “If a human had written the paper and submitted it to the Annals of Mathematics and I had been asked for a quick opinion, I would have recommended acceptance without any hesitation. No previous AI-generated proof has come close to that.”

The proof illustrates a general trend towards autonomous, agentic problem-solving in AI systems. OpenAI describes the system that produced the proof as a general-purpose model not specialized in mathematics. AIs can now perform long, novel chains of reasoning on difficult problems and are beginning to outstrip our ability to measure their progress.

AI agents still perform best in domains with easily verifiable outputs, such as mathematics and cybersecurity. For example, Anthropic's Claude Mythos found thousands of vulnerabilities across every major operating system and web browser, and was deemed too dangerous for public release. Such capabilities are why the government is now more interested in evaluating frontier AI models.

AI research is also a field with many easily verifiable outputs. Researchers at OpenAI and Anthropic take advantage of this fact to accelerate their work; senior researchers now claim they make only high-level decisions and let AI handle most of the coding. Experimenting with the coding capabilities of a publicly available AI system, like Claude Code, immediately demonstrates how far AI has come in the last year.

OpenAI and Anthropic intend to use AI to enhance future models with minimal human oversight. To justify the urgency, these companies cite the importance of beating rival U.S. or Chinese labs. Many of the field’s foremost experts warn that this race ends with human extinction.

Policymakers and researchers, including the founders of the AI revolution, are calling for international restrictions on the technology. A growing bipartisan and international consensus of political leaders agree.

[AI StopWatch @AIStopWatch]

In today's Digest:

* OpenAI, Anthropic, SpaceX race to file IPO.
* The AI executive order is postponed.
* METR evaluates rogue deployment risks.
* AI makes a breakthrough in mathematics.

[MIRI @MIRIBerkeley]

New paper from MIRI data scientist @robi_rahman: Does Distributed Training Undermine Compute Governance?

[Robi Rahman @robi_rahman]

1/ With distributed training, you could violate an AI pause treaty by training a GPT-4-scale model over consumer internet, using hardware below every proposed compute governance threshold, for under $100M. My new paper in @taig_icml explains how to catch this and shut it down.

[Center For Humane Technology @HumaneTech_]

The wait is over! Starting today, May 15, you can stream @theaidocfilm on @peacock.

This film takes the dizzying complexity of AI — the promise, the peril, the competing ideologies, the economic incentives — and creates a shared experience we can all see and respond to.

Then, after you've watched, head to https://t.co/AQ5JkWwrOO to explore concrete actions we can all take to build a better future with AI.

[MIRI @MIRIBerkeley]

For more, check out the paper from MIRI's Technical Governance Team. https://t.co/nbq900VuLC

[MIRI @MIRIBerkeley]

For an agreement like this to be effective, both the US and China would need to be party to it.

Here’s one plausible path to that outcome, in six stages 🧵

[Peter Barnett @peterbarnett_]

We at the MIRI Technical Governance Team just put out a report describing an example international agreement to prevent the creation of superintelligence. 🧵

[MIRI @MIRIBerkeley]

Stage Six: creating superintelligence, after the necessary solutions have been discovered to do so without posing an unacceptably high danger to the world.