Attending #TOKEN2049? 🇸🇬
Then don't miss the #deAISummit panel discussion, "What Can Blockchain Bring to AI?" 🤖
Understand the synergy between blockchain and AI and how these technologies are revolutionizing various sectors. 🔥
Reserve your spot 👇
https://t.co/hwPrg6poJ1
o1 is the biggest AI update since transformers and it paves a clear path toward AGI.
This is what Ilya saw. This is why Sam wanted trillions of dollars of compute. This is what Leopold warned us about.
Just like transformers, o1 lets you efficiently trade in compute for accuracy, but here it's at inference time instead of pretraining time. It's the AlphaGo moment for transformers. Every big lab was working on this, OpenAI was the first to release.
The path toward AGI is now clear. Train this over images, video, web interactions, robotics sensory data, every source of data we got. If you can construct an RL reward function for a task -- easy to do for code and math -- you can now get better at that task by throwing more RL + inference compute at it.
OpenAI introduced a remarkable new log scaling law, for inference. This is now the most important scaling law. They'll proceed to scale this stuff up 10, 100, 1000x. Sure there might be a hiccup here or there as power plants become harder than compute to spin up and regulations restrict the flow of research. But the benefits are so big and the national implications so grave that this isn't going to stop.
I'm super hyped. I grew up dreaming of a sci-fi future -- didn't you? -- and this is the way to get there. I want an iron man suit, pristine cities, a millennium of feeling 27 years old, trips to Europe that take 1 hour and trips to Europa that take 1 year.
Humans are too slow, lazy, and political to reach this future on our own. We'd sooner wipe ourselves out. Plentiful intelligence is how we get to this future safely.
Sure there are risks, but one other remarkable result from OpenAI's report is that o1 actually gets better at following rules. I'm less worried that the AI itself will choose chaos, and far more worried that bad actors will get access to this tech and RL it toward chaos. We should obviously march forward carefully.
It's a strange feeling to have been planning for this agentic future for a couple years, and to now see it finally arriving. Greg probably doesn't remember this, but I was once chilling with him on a beanbag in the GPT-2 days at an OpenAI WestWorld watching party, and we were arguing about scaling laws. I thought we needed new algorithms to get to AGI. He argued scale is all you need. After GPT-3, I felt he was right. After GPT-4, I knew it. Now with o1, I'm prepared for it.
The era of agents has officially begun, and me and my team are ready. OpenAI's only a block away from our office, and we feel a parallel energy.
Agents powered by lots of inference are going to change our society rapidly. The stakes could not be higher. We need lots of different kinds of help to navigate this well. My plan is to provide these agents with super powerful web retrieval, bc no one is focusing on that and it's extremely important to get right. What's your plan?
Inference-time compute scaling is the latest buzzword in AI.
The idea here is to get to the next level of AI performance improvements by increasing the compute used during inference, rather than focus on training larger AI models.
This paper demonstrates that:
- Scaling inference compute through repeated sampling leads to large improvements across a variety of tasks.
- Suggests that there is an exponential scaling law behind it.
If true — the demand for GPUs will ramp up massively soon.
We are not ready for this future.
Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
https://t.co/CGFPBGn7ST
🚀 @cot_research Co-Founder @0xPrismatic will moderate the panel ''What can blockchain bring to AI" panel at the deAI Summit during #TOKEN2049!
🔥 Reserve your spot now: https://t.co/hwPrg6poJ1
#deAIsummit