Daniel

Just an old programmer. Interested in mountain bike rides, photography, and some research on AI.

[Dwarkesh Patel @dwarkesh_sp]

New blackboard lecture w @ericjang11

He walks through how to build AlphaGo from scratch, but with modern AI tools.

Sometimes you understand the future better by stepping backward. AlphaGo is still the cleanest worked example of the primitives of intelligence: search, learning from experience, and self-play. You have to go back to 2017 to get insight into how the more general AIs of the future might learn.

Once he explained how AlphaGo works, it gave us the context to have a discussion about how RL works in LLMs and how it could work better – naive policy gradient RL has to figure out which of the 100k+ tokens in your trajectory actually got you the right answer, while AlphaGo’s MCTS suggests a strictly better action every single move, giving you a training target that sidesteps the credit assignment problem. The way humans learn is surely closer to the second.

Eric also kickstarted an Autoresearch loop on his project. And it was very interesting to discuss which parts of AI research LLMs can already automate pretty well (implementing and running experiments, optimizing hyperparameters) and which they still struggle with (choosing the right question to investigate next, escaping research dead ends). Informative to all the recent discussion about when we should expect an intelligence explosion, and what it would look like from the inside.

Timestamps:

0:00:00 – Basics of Go
0:08:06 – Monte Carlo Tree Search
0:31:53 – What the neural network does
1:00:22 – Self-play
1:25:27 – Alternative RL approaches
1:45:36 – Why doesn’t MCTS work for LLMs
2:00:58 – Off-policy training
2:11:51 – RL is even more information inefficient than you thought
2:22:05 – Automated AI researchers

[annie @_annieversary]

what the Fuck https://t.co/jSWRRiEBQ5

[Daniel @daniel_berns]

Pay per fail

[Shishir @ShishirShelke1]

Artemis II crew is thousands of miles away from Earth

And they’re asking ground crew for help because they have two versions of Microsoft Outlook open and neither is working

This scene is now canon 😭

[𝚃𝙷𝙴 𝚆𝙷𝙸𝚃𝙴 𝚁𝙰𝙱𝙱𝙸𝚃 @White_Rabbit_OG]

Finally 👏👏👏

[Turing Post @TheTuringPost]

A useful survey – "Anatomy of Agentic Memory"

Explains why agent memory systems often fail in practice, focusing on how systems store and manage information over long interactions

Covers:

- Memory-Augmented Generation (MAG)
- Agent memory architectures - lightweight semantic, entity-centric/personalized, episodic & reflective, structured/hierarchical
- Benchmark saturation and metric problems
- Backbone dependence
- LLM-as-a-judge instability
- System costs: latency, retrieval overhead, throughput

[Andrej Karpathy @karpathy]

New art project.
Train and inference GPT in 243 lines of pure, dependency-free Python. This is the *full* algorithmic content of what is needed. Everything else is just for efficiency. I cannot simplify this any further.
https://t.co/HmiRrQugnP

[Daniel @daniel_berns]

@snaut66 @bayraktar_1love Ukrainian "space debris"?

[Daniel @daniel_berns]

@LucasSa56947288 Holding hands?

[Juan Francisco @medpedmex]

@Farmaenfurecida https://t.co/u7X8Vb8scr

[Daniel @daniel_berns]

@NVIDIAGeForce GeForce Day

[Sakana AI @SakanaAILabs]

What if we could evolve AI models like organisms in nature, letting them compete, mate, and combine their strengths to produce ever-fitter offspring?

Excited to share our new work: “Competition and Attraction Improve Model Fusion” presented at GECCO’25🦎 where it was a runner-up for best paper!

Paper: https://t.co/ihfdriOPNw
Code: https://t.co/lOW2ghb5bj

Summary of Paper

At Sakana AI, we draw inspiration from nature’s evolutionary processes to build the foundation of future AI systems. Nature doesn’t create one single, monolithic organism; it fosters a diverse ecosystem of specialized individuals that compete, cooperate, and combine their traits to adapt and thrive. We believe AI development can follow a similar path.

What if instead of building one giant monolithic AI, we could evolve a whole ecosystem of specialized models that collaborate and combine their skills? Like a school of fish 🐟, where collective intelligence emerges from the group.

This new paper builds on our previous research on model merging, which follows such an evolutionary path. We started by using evolution to find the best “recipes” to merge existing models (our Nature Machine Intelligence paper: https://t.co/zqs7kGL4Gl). Then, we explored how to maintain diversity to acquire new skills in LLMs (our ICLR 2025 paper: https://t.co/00Buf1051A). Now, we're combining these ideas into a full evolutionary system.

A key limitation remained in earlier work: model merging required manually defining how models should be partitioned (e.g., by fixed layer or blocks) before they could be combined. What if we could let evolution figure that out too?

Our new paper proposes M2N2 (Model Merging of Natural Niches), a more fluid method, which overcomes this with three key, nature-inspired ideas:

1/ Evolving Merging Boundaries 🌿: Instead of merging models using pre-defined, static boundaries (e.g. fixed layers), M2N2 dynamically evolves the “split-points” for merging. This allows for a far more flexible and powerful exploration of parameter combinations, like swapping variable-length segments of DNA rather than entire chromosomes.

2/ Diversity through Competition 🐠: To ensure we have a rich pool of models to merge, M2N2 makes them compete for limited resources (i.e., data points in a training set). This forces models to specialize and find their own “niche,” creating a population of diverse, high-performing specialists that are perfect for merging.

3/ Attraction and Mate Selection 💏: Merging models can be computationally expensive. M2N2 introduces an “attraction” heuristic that intelligently pairs models for fusion based on their complementary strengths—choosing partners that perform well where the other is weak. This makes the evolutionary search much more efficient.

Does it work?

The results are fascinating: This is the first time model merging has been used to evolve models entirely from scratch, outperforming other evolutionary algorithms. In one experiment, starting with random networks, M2N2 evolved an MNIST classifier that achieves performance comparable to CMA-ES, but is far more computationally efficient.

Does it scale?

We also showed that M2N2 can scale to large, pre-trained models: We used M2N2 to merge a math specialist LLM with an agentic specialist LLM. M2N2 produced a merged model that excelled at both math and web shopping tasks, significantly outperforming other methods. The flexible split-point was crucial here.

Does it work on multimodal models?

When we applied M2N2 to text-to-image models, we merged several models by adapting them only for Japanese prompts. The resulting model not only improved on Japanese but also retained its strong English capabilities—a key advantage over fine-tuning, which can suffer from catastrophic forgetting.

This nature-inspired approach is central to Sakana AI’s mission to find new foundations for AI based on collective intelligence. Rather than scaling monolithic models, we envision a future where ecosystems of diverse, specialized models co-evolve, collaborate, and combine, leading to more adaptive, robust, and creative AI. 🐙

We hope this work sparks more interest in these under-explored ideas!

Published in ACM GECCO’25: Proceedings of the Genetic and Evolutionary Computation Conference. DOI: https://t.co/5eSwhvs5tQ

[Quanta Magazine @QuantaMagazine]

A team of mathematicians just built the first “monostable” tetrahedron. Someday, it might help inform the design of a self-righting spacecraft.
https://t.co/MshP0f8fgk

[Daniel @daniel_berns]

@pickover 1) a * (100 + 10 + 1) / (3 * a) = b
then b = 111/3 = 37.

[Google AI Developers @googleaidevs]

YouTube tutorial + Gemini CLI

[Gary Marcus @GaryMarcus]

BREAKING: Explosive new paper from MIT/Harvard/UChicago.

Things just got worse — a lot worse — for LLM’s and the myth that they can understand and reason.

The paper documents a pattern they called Potemkins, a kind of reasoning inconsistency (see figure below). They show that LLMs - even models like o3 — make these errors frequently.

You can’t possibly create AGI based on machines that cannot keep consistent with their own assertions. You just can’t.

“success on benchmarks only demonstrates potemkin understanding: the illusion of understanding driven by answers irreconcilable with how any human would interpret
a concept … these failures reflect not just incorrect understanding, but deeper internal incoherence in concept representations”

Game over for any hopes of building AGI on a pure LLM substrate. cc @geoffreyhinton, checkmate.

[Daniel @daniel_berns]

Nice article for the weekend

[Sergey Levine @svlevine]

Self-supervised representation learning looks a bit like RL. What if we literally use RL as a SSL method for visual representations? Turns out that it works quite well. In new work by @its_dibya, we show how this can be done: https://t.co/JTTbqIW2Pv

[Sergey Levine @svlevine]

Self-supervised representation learning looks a bit like RL. What if we literally use RL as a SSL method for visual representations? Turns out that it works quite well. In new work by @its_dibya, we show how this can be done: https://t.co/JTTbqIW2Pv

[Daniel @daniel_berns]

About detection of emerging system of values in LLM.

[Ethan Mollick @emollick]

https://t.co/7eNytkUmIp

[Daniel @daniel_berns]

Design principles for agents

[Jerry Liu @jerryjliu0]

We’re excited to release an interactive guide highlighting the definitive set of principles for building AI agents 🔥

Based on the popular 12-Factor agents repo by @dexhorthy. We packaged the principles into an interactive website and Colab notebook with working code examples, so that you can incorporate these principles into your agent application in minutes.

As a refresher - some of the principles include getting structured outputs from tools, state management, checkpointing, human-in-the-loop, error handling, composing smaller agents into bigger ones, and more.

This is amazing work by @seldo and should be required reading for anyone looking to get started building agents.

Website: https://t.co/4F1yt37OwS
Notebook: https://t.co/jiNCzQt05H

[Daniel @daniel_berns]

Paper about reasoning with multimodal input.

[Xin Eric Wang @xwang_lk]

Project scooped by OpenAI? No worries! Proprietary LLM methods are behind the door and not the only way forward.

Introducing GRIT (Grounded Reasoning with Images & Text): Teach Multimodal LLMs to think with images for general visual reasoning tasks, with just 20 examples! Pure RL, no supervised fine-tuning, no labeled reasoning data.

Simpler, efficient, and fully open-source. 🚀 #OpenSource #MLLM #reasoning