Introducing Hyperagents: an AI system that not only improves at solving tasks, but also improves how it improves itself.
The Darwin Gödel Machine (DGM) demonstrated that open-ended self-improvement is possible by iteratively generating and evaluating improved agents, yet it relies on a key assumption: that improvements in task performance (e.g., coding ability) translate into improvements in the self-improvement process itself. This alignment holds in coding, where both evaluation and modification are expressed in the same domain, but breaks down more generally. As a result, prior systems remain constrained by fixed, handcrafted meta-level procedures that do not themselves evolve.
We introduce Hyperagents – self-referential agents that can modify both their task-solving behavior and the process that generates future improvements. This enables what we call metacognitive self-modification: learning not just to perform better, but to improve at improving.
We instantiate this framework as DGM-Hyperagents (DGM-H), an extension of the DGM in which both task-solving behavior and the self-improvement procedure are editable and subject to evolution. Across diverse domains (coding, paper review, robotics reward design, and Olympiad-level math solution grading), hyperagents enable continuous performance improvements over time and outperform baselines without self-improvement or open-ended exploration, as well as prior self-improving systems (including DGM). DGM-H also improves the process by which new agents are generated (e.g. persistent memory, performance tracking), and these meta-level improvements transfer across domains and accumulate across runs.
This work was done during my internship at Meta (@AIatMeta), in collaboration with Bingchen Zhao (@BingchenZhao), Wannan Yang (@winnieyangwn), Jakob Foerster (@j_foerst), Jeff Clune (@jeffclune), Minqi Jiang (@MinqiJiang), Sam Devlin (@smdvln), and Tatiana Shavrina (@rybolos).
The origin of all human knowledge is not sensory data nor is it an extrapolation of the future from the past.
Our knowledge consists of bold, creative guesses.
"A lot of the interesting things in life are explaining the scene in terms of the unseen."
~Conjecture Institute Founding Donor @naval
(video script by Fellow @arjunkhemani & President @ChipkinLogan)
The “unifying concept” allowing people to understand everything is a basic idea: explanatory universality. But, as I have learned, “basic” does not mean “simple”.
I have spent many years now making use of this idea - that people are “universal explainers” - to pick apart a variety of problems. Rather often, my interlocutors seem a mixture of puzzled and frustrated about my focus on that basic idea. It has, therefore, at times puzzled and frustrated me that others do not see the significance of explanatory universality.
This book is my attempt to remedy some of the confusion.
~Conjecture Institute Ambassador @ToKTeacher
There’s just way more incentive to think from a new story. I don’t feel satisfied ruminating over things endlessly anymore. It doesn’t make sense to me, knowing how that circumstance itself was my creation and I don’t have to live in it for another second if I don’t want to
When you say that something is possible or impossible, you're not directly referring to likelihood or probabilities for it to happen.
Probabilities are not there in the foundations of constructor theory.
~Conjecture Institute Senior Scientist C. Marletto w/ @TOEwithCurt
Even if I see this as a scam, AI appearing human-like and conscious is still possible. What matters isn’t whether AI truly has consciousness, but whether people think it does.
FEP is likely the closest scientific explanation for the LoA. While the brain typically minimizes free energy by conforming to sensory evidence, LoA requires the opposite. Instead of updating the model to fit the world, use active inference to hold assumption until 3D aligns.
Life is not a struggle against entropy. A living organism dies if its entropy becomes too low as surely as if it becomes too high.
Yes, an organism takes in ordered energy and voids disordered energy to the outside. But so does a rock in the sun. That isn't what life is about.
Like @davidbessis and others, I think that Hinton is wrong. To explain why, let me tell you a brief story.
About a decade ago, in 2017, I developed an automated theorem-proving framework that was ultimately integrated into Mathematica (see: https://t.co/nGCIUk44TP) (1/15)
The Joy of Cryptography is finally on bookshelves! Thanks to @mitpress for supporting the adventure. The book will eventually be completely open-access, but for now the first 3 chapters are available at https://t.co/gsiUIjWGim, featuring some fun interactive elements.
human biology literally self-terminates without love. it also self-terminates in acrimony.
human biology thrives and even rejuvenates itself in the elixir of love.