i think some people are hoping that self-distillation enables “exploration-free” RL purely via reflection on live data, allowing them to bypass the need for replayable environments
unfortunately, RL is all about exploration
my instinct is you basically need to model the world
over the weekend i checked the obvious thing, which is whether mythos is able to solve the erdos unit distance problem, aka erdos problem #90. the answer is: yea
I regret that comment, which was less polite than I aim to be on here. Let me try to write something a bit more substantial.
I don't think "discover all math that can possibly be discovered" is really coherent. Even if one conceptualizes mathematics as "answering well-posed open problems," historically resolutions to such have tended to raise more questions--I would argue that there are now more interesting open questions than there have been at any time in the past, despite (in fact because of!) the fact that there are more mathematicians than ever, resolving more problems than ever.
At any fixed capability level I think it is likely we will see lots of problems remain open, including many basic open problems we know about at present. It's much easier to pose an interesting question than to answer one; it seems to me that the difficulty of interesting questions we can generate is basically unbounded.
But also mathematics consists of much more than this--less verifiable tasks include things like "understand such-and-such an object," or "find a cool phenomenon," or "develop a theory," though such tasks are often benchmarked by their impact on problem-solving. I think AI will eventually (perhaps soon) be able to do these kinds of things but the idea that it will exhaust the supply of math, or that we won't want to develop human capital to understand some portion of what is discovered, seems to rely on an understanding of math and our motivations for working on it that is, at least, alien to me.
I've recently got in on the act of getting AI to solve open problems in mathematics. More precisely, I gave some questions asked by Melvyn Nathanson to ChatGPT 5.5 Pro, to which I have been given access, and it answered them. 🧵
The Jensen Huang episode.
0:00:00 – Is Nvidia’s biggest moat its grip on scarce supply chains?
0:16:25 – Will TPUs break Nvidia’s hold on AI compute?
0:41:06 – Why doesn’t Nvidia become a hyperscaler?
0:57:36 – Should we be selling AI chips to China?
1:35:06 – Why doesn’t Nvidia make multiple different chip architectures?
Look up Dwarkesh Podcast on YouTube, Apple Podcasts, Spotify, etc. Enjoy!
I think one of the conclusions we should draw from the tremendous success of LLMs is how much of human knowledge and society exists at very low levels of Kolmogorov complexity.
We are entering an era where the minimal representation of a human cultural artifact... (1/12)
The @ilyasut episode
0:00:00 – Explaining model jaggedness
0:09:39 - Emotions and value functions
0:18:49 – What are we scaling?
0:25:13 – Why humans generalize better than models
0:35:45 – Straight-shotting superintelligence
0:46:47 – SSI’s model will learn from deployment
0:55:07 – Alignment
1:18:13 – “We are squarely an age of research company”
1:29:23 – Self-play and multi-agent
1:32:42 – Research taste
Look up Dwarkesh Podcast on YouTube, Apple Podcasts, or Spotify. Enjoy!
Yes.
An easy recent example is using AI to write a fast Fourier transform in Rust.
I don’t know rust and I don’t know how to write a fast Fourier transform.
There is no world in which I would have learned how to do that “on the job” for the problem I was trying to solve.
So it either would have required ~infinity time, or hiring the kind of person who can do that (easy $300k total comp) rather than the ~$3-5 in tokens and 15 minutes that were required by AI.
The Naroditsky family shares the sad news of Daniel’s unexpected passing. Daniel was a talented chess player, educator, and beloved member of the chess community. We ask for privacy as the family grieves.