@marcoarment Didn’t know you played Total Annihilation! That game was incredibly impactful on me growing up. So, so good. I wasn’t a part of the online community though…for me that was Homeworld a little bit later 😄
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
https://t.co/zHij5zwKGT
There's been a ton of talk about the role of humans in code review, and when and how humans should be signing off on changes.
I believe that, long-term, humans have no role in routinely reviewing code.
I believe the demand for compute is going to go up faster than the supply of compute, so the price of compute is going to increase significantly in the future, perhaps as much as 10x in the next few years. I think it is a very good deal to buy compute in the year 2030 at the prices of 2026.
This is a very important trend: tokens in the future will be worth more than tokens are worth today, because models in the future will be much smarter than models today. The value of tokens from the best models in 2028 will be several times the value of a tokens today. As models become more intelligent, they can do more difficult work, and the value of the work they do goes up. So I believe that the residual value of GPUs, i.e. their resale value, is under-appreciated.
Inference demand today is bottlenecked by dumb agents that require human intervention. Smart agents of the future will require very little human intervention. That doesn't just mean that we have the same number of agents running for a longer period of time. It means we will have 10x, 100x, 1000x more agents, because one human can be responsible for thousands or hundreds of thousands more agents. In other words, it's too simplistic to look at market penetration of ChatGPT today and predict future compute usage by assuming market penetration increases and multiplying by that amount. You have to account for the fact that each user will use a lot more AI; there is a whole other multiplicative factor.
Furthermore, there are several application areas which will be compute-hungry and have yet to fully develop or be deployed. Each of robotics, AI trading, and AI for materials science and biotech, represent at least an entire agentic-coding sector equivalent of compute demand, if not more, in the medium-term. Even if AI fails to take off in many of these new verticals tomorrow, for which billions of venture capital is being deployed into today, breakout demand in just one would qualify an enormous increase in compute demand.
These are my predictions, and of course there is risk and uncertainty, but I think that most people are underrating how much more demand is going to increase.