Over the last few months, I have been thinking a lot about what RSI and superintelligence mean for the scientific world that I live in. The trigger for putting down my thoughts on paper was @dwarkesh_sp's fascinating, thought-provoking essay on RSI.
The piece is a little long but contains pretty much the condensed sum total of my (always evolving) thoughts until now. I start out with skepticism but end with optimism since I believe the possibilities are extraordinary. Hope you read it and take something away from it. Here's a bulleted summary:
1. Recursive improvement in AI is not necessarily recursive improvement in science. A model that becomes 10× better at designing its successor doesn't automatically become 10× better at designing proteins, drugs, or materials. Today's AI agents still depend on imperfect scientific tools such as AlphaFold, RFdiffusion, and molecular simulations. Whether increasingly intelligent models can overcome these limitations is an open hypothesis.
2. Mathematics and software allow rapid, often unambiguous verification. Biology, chemistry, and medicine depend on experimental evidence, physical timescales, and emergent complexity. Intelligence can dramatically accelerate scientific discovery, but it cannot always substitute for new information from the physical world. I discuss, from the history of science, examples where the real leap was interrogating the world in entirely new ways, a process that involved not just intelligence but courage.
3. The possibilities are nevertheless extraordinary. AlphaFold demonstrates how AI can extract remarkable insights from existing scientific data. Even more exciting is the prospect of AI systems that improve not just their reasoning and scientific models, but the experimental instruments themselves - inventing better assays, more sensitive measurements, faster laboratory techniques, more versatile robotics, and entirely new ways of interrogating nature.
Coupling increasingly intelligent AI to increasingly powerful scientific instruments could create genuinely recursive loops of discovery, in which better experiments produce better data, better data produce better models, and better models enable still better experiments. This is an exuberant future to look forward to.
My central argument: Superintelligence is not, by itself, an adequate theory of scientific progress. But when intelligence is coupled to experiments and empirical feedback, the possibilities are breathtaking.
https://t.co/mH3fpMQqUk
ten chemical reactions that changed the course of human history.
from feeding billions of people to the invention of plastics, these reactions shaped modern civilization.
which one do you think had the greatest impact?
This is all very true. Of course, the one thing we know about science is that solving any or all of these questions will only create more questions. Whether tackled by AI or human, that is why science will always remain an exciting open frontier.
everyone is afflicted by the great malaise of ignorance. I’m not sure whether P=NP or not. I don’t know why the universe is expanding at an increasing rate. it’s a great thing to reduce ignorance in the world, no matter if man or machine accomplishes it
This is false. Alpoge's proof indeed winds its way through a dizzying array of "known" mathematics. But known is not the same as "elementary". The mathematical objects and constructions used in the proof are extraordinarily sophisticated, spanning almost every subfield of number theory.
Even an incredibly long and complicated elementary proof requires no advanced mathematical knowledge to understand, at least *locally*. So any competent mathematician, given enough time, could in principle understand all the parts of an elementary proof. Alpoge's proof, by contrast, draws on such a vast range of sophisticated mathematics, much of it developed over the past 50 years, that there may well be no single mathematician alive with the breadth of expertise needed to understand *all* its parts.
One thing that I think the "AI will solve biology" people fail to appreciate is that there is nothing like Lean for biology; you just have to literally run the experiments.
And *chemical* space is unfathomably large, let alone complete biological organisms within an environment.
It’s so sad to open twitter and just read nonsense.
We built the greatest information-sharing technology in history, only to reward those who make people angrier instead of smarter.
I often post my favorite history books, but these were my father's favorite history books - he was even more of a historical tome reader than I am. Both history and historical fiction were his favorites, and in particular he had read almost all of James Michener's epic books.
@JeffDean Imagine fleeing political persecution for America and facing this; how far from John Winthrop and Reagan's shining city upon a hill have we fallen. Thanks for your advocacy; it's having influential people like yourself speak that makes the difference.
Running some triple-zeta DFT and coupled cluster calculations on exactly the kind of small but highly informative model chemical system that has warmed my heart in the past. Codex and Runpod GPUs make it a breeze.
In these Riemann-hypothesis-fueled uncertainty, the physicist in me is for the first time hopeful that, in my lifetime, I might see a reconciliation of quantum mechanics and general relativity.
I've never known the tech world to be more unanimous about anything than they are about Jensen Huang. Everyone in tech thinks the board made a huge mistake. Except the board. But they are not in tech.
@Danny_Ruiz123 You are right. That cyclopropane carbon should not be chiral since the fused system has a horizontal mirror plane of symmetry. Good observation.
Funny behavior: 5.6 Sol didn't just pick a smaller ring, it broke some bonds and created a new macrocycle (this is the RevMed pancreatic cancer drug, by the way). In my own work I have used 6 Astra for perhaps fifty medicinal chemistry workflows, and the structures have been remarkably accurate.
MedChemBench is a new benchmark for AI reasoning in medicinal chemistry.
It shows the progress our frontier models have made in their ability to help scientists use AI to identify promising drug candidates.
https://t.co/4N1H77z2YU