Xiaohu Zhu ⏹️ CSAGI

The first who gave alignment the Chinese translation, 对齐.
· Global AI Treaty @BaruchPlanForAI · @Foresightinst · @FLIxrisk

[Xiaohu Zhu ⏹️ CSAGI @neil_csagi]

Current embarrassment: all AI labs have the "pseudo preparedness".

[Seth Lazar @sethlazar]

One of the most ritualistic phrases in the responsible AI (etc) field over the last decade has been the sanctimonious observation that *of course* all reasons ultimately apply to people, not to the AI systems themselves. Similarly, whenever AI exercises power, the catechism required you to say that it did always so *on behalf* of some human/s, never itself. I think this is probably false already; if not, it's only a matter of time.

[Rosie Campbell @RosieCampbell]

The deadline for this is June 21! Please share with anyone who might be interested in contributing to empirical work on digital minds

[Jason Weston @jaseweston]

Our co-improvement position paper is now on arXiv!
(We've updated it, covering more existing work.)
📝: https://t.co/xnxWYoMNP7

After >27 years of research, my first position paper!

Short 🧵 (1/5) follows 👇

Synopsis: it's about building AI that collaborates on AI research *with us* to solve AI faster, and to help fix the alignment problem together.

How? Build the AI with those collab skills (i.e., we create benchmarks! training data! methods! etc. for that).

I've been personally inspired by @Yoshua_Bengio's recent talks on safety & AI research, and also from seeing Nicholas Carlini's COLM keynote where he said we researchers can all do our bit to help (paraphrased). So – hope this helps! 🙏

[Eliezer Yudkowsky ⏹️ @ESYudkowsky]

On a first read, this paper seems far ahead of the pack in terms of (1) understanding some reasons why a task might stay difficult even in the face of gradient descent, and (2) distilling out propositions they'd need to somehow verify before they started expecting nice things.

[Xiaohu Zhu ⏹️ CSAGI @neil_csagi]

@lunarwallfacer_ Can you explain it?

[Xiaohu Zhu ⏹️ CSAGI @neil_csagi]

Phase transition in alignment community. Safe AGI Game continues.

[Geoffrey Irving @geoffreyirving]

We are starting a new, nonprofit alignment organization, ⊢ Sequent Research, bringing together researchers previously on UK AISI’s Alignment Team, Timaeus, and elsewhere to research how to align superintelligence. We are hiring! 🧵

[Xiaohu Zhu ⏹️ CSAGI @neil_csagi]

@geoffreyirving Congrats!

[Cas (Stephen Casper) @StephenLCasper]

According to the MIT Libraries' database of theses (dating back to the 1800s), my thesis was only the 2nd in the institute's history to contain the word "shit."

[Eliezer Yudkowsky ⏹️ @ESYudkowsky]

Today is June 5th, one day to take a break from fighting each other online, and remind ourselves of our shared humanity and common goals by uniting around the one thing we all agree about: Repealing the Jones Act. https://t.co/ajQqtTdt6z

[Holly ⏸️ Elmore @ilex_ulmus]

https://t.co/ogP7cjc3dc

[Xiaohu Zhu ⏹️ CSAGI @neil_csagi]

What I cannot stop, I do not understand. https://t.co/oBOGwc3NXd

[Shane Legg @ShaneLegg]

A great interview with our new Director of AGI Economics @alexolegimas and the economist @pawtrammell. It's a good antidote to some of the overly simplistic narratives about the economics of AGI.

[Brian Christian @brianchristian]

Just published in @PNASNews, we resolve a 50-year-old riddle from Richard Feynman's handwritten notes, prove and generalize it, and run a large-scale human study to reveal near-optimal heuristics in sequential decision problems:

https://t.co/4AOM1iDqG2

[Xiaohu Zhu ⏹️ CSAGI @neil_csagi]

@DavidSKrueger This episode was recorded on December 4, 2025.

[Tiberiu Mușat @Tiberiu_Musat_]

Why does deep learning generalize? What does weight decay really do? Can algorithmic information theory address these questions?

In my latest preprint, I give a proof that the minimum neural weight norm matches the minimum program length (aka Kolmogorov Complexity), up to a logarithmic factor. In other words, the neural network with the smallest possible weight norm (that fits the data) must encode the shortest program (that fits the data).

The result only holds for fixed-precision neural nets: infinite precision nets can store infinite information with finite (small) weights.

https://t.co/eMZIGQDf2f

[Elizabeth Barnes @BethMayBarnes]

Sometimes people outside the field say things like “The AI situation can’t be that bad, there must be experts who are on top of it”. As “an expert”, I would like to be clear that we are *not* on top of it. Some key aspects of the situation IMO:

[Goodfire @GoodfireAI]

The most popular way to interpret AI is missing the bigger picture.

Models think in curved shapes. But sparse autoencoders (SAEs) work with straight lines.

Can they still capture models’ curved neural geometry? Yes, but not how you might think! (1/7)

[steve hsu @hsu_steve]

AI Billionaire on Existential Risk: Jaan Tallinn, Manifold episode #112

Jaan Tallinn is a tech billionaire and founding engineer of Skype who leverages his wealth to mitigate existential risks from artificial general intelligence (AGI). He co-founded the Future of Life Institute and the Centre for the Study of Existential Risk, while making early foundational investments in frontier AI labs like DeepMind and Anthropic.

0:00 Assessing Current AI Risk Levels
02:11 Self-Sustaining AI Scenarios
07:55 Global AI Race Dynamics
41:18 Explaining the Techno-Capital Flywheel
46:45 Insider Origins of AI Safety
55:42 Race Politics and Public Fear
01:25:45 Pop Culture, Movies, and Fame
01:30:23 Big Questions for Humanity's Future

[Alec Helbling @alec_helbling]

The Helmholtz decomposition is one of the fundamental results of vector calculus.

It says any well-behaved vector field can be split into two parts, one capturing sources and sinks through divergence, and one capturing rotation through curl.

[Dr. Roman Yampolskiy @romanyam]

2nd episode of The Roman Forum is an interview with AI Safety/Governance expert Connor Leahy @NPCollapse. Connor is a great speaker and is lobbying to get government to ban Superintelligence.

My first virtual recording. Got a good mic, should probably hold it closer. Lots of room for improvement, but once I get familiar with software/hardware/setup things will feel a lot more natural. Enjoy and subscribe!