The context problem in AI is much deeper than it looks.
Feeding an LLM more documents doesn't solve it. Context isn't just information. It's understanding built from experience over time. The only real fix is continual learning: AI that keeps updating from the environment it operates in, rather than freezing at training time.
At Skyfall, this has shaped our research roadmap from the start.
First, we explored world models as planners (worth reading Dr. Fei-Fei Li's recent breakdown on this). Our work on SCOPE showed that a small, specialized world model could outperform frontier LLMs on sequential decision-making by being 55x faster than GPT-3.5, 160,000x smaller than GPT-4o, and more accurate on planning tasks.
Now we're focused on world models as simulators. A simulator isn't just a rendering of the world. It's a physically and dynamically faithful environment that agents can actually train in. And for a simulator to stay useful over time, it needs continual learning at its core. Agents that train in a static simulation will eventually hit a ceiling. Agents that train in a world that evolves with them won't.
Context is a learning problem. That's what we're building toward, and we're launching something soon. Stay tuned. ✨
Language models gave AI the ability to talk about the world. World models will give AI the ability to understand it. But “world model” is an overloaded term. What does it mean? HAI Founding Director @drfeifei offers the taxonomy that matters now. https://t.co/n7Shl5v3wT
@StanfordHAI@drfeifei Great to see this conversation going mainstream. This has been central to our research at Skyfall for a while now. Check our research on world models as planners here: https://t.co/Rs4l9Tw1pO
For those who want to go deeper, here's our research so far:
- LLMs playing OpenRCT2: https://t.co/H7cqkCeAeW
- LLMs and Enterpise Workflows: https://t.co/n1cdJdzp9v
- SCOPE, neural planner: https://t.co/n3DDSqsPGU
- CASSANDRA: https://t.co/kReoPn2Kxw
- AI CEO: https://t.co/KPipIW8Kpa
LLMs taught us what AI could do with language, but language isn't enough.
The next layer is world models. Systems that don't just predict text but understand space, causality, and consequence. Systems that know what happens when something changes, not just how to describe it.
➡️That's what we're building at Skyfall.
@PalantirTech The ontology insight applies far beyond agriculture. Any domain where data is fragmented, ungoverned, and high-stakes needs the same foundation.
The context point is right, but it actually proves the opposite of AGI.
A truly intelligent system wouldn't need context handed to it. It would learn context on its own, continuously, from the environment it operates in. The fact that today's AI can't do that is exactly why we haven't achieved AGI.
That's the continual learning gap - something our team is working hard on!
Guillermo's framing here is spot on. To add to it: the most valuable infrastructure for agents isn't just reusable code. It's systems that understand the environment they operate in and get smarter over time.
World modeling and continual learning are what turn a building block into a foundation. That's what we're building at Skyfall.
"Faster than we thought" is doing a lot of work here.
Speeding up training runs is optimization, not self-improvement. True RSI requires a system that learns continuously from new experience without forgetting what it knows. That's continual learning 👀, and we're nowhere near cracking it.
@BernardMarr This is what we are working toward at @skyfallai
We elaborated more on our research and what enterprises require next here: https://t.co/latsX06BT8
@ionleu https://t.co/CKsCAhoGvY ➡️ Reimagining enterprise software with world modeling and continual learning
Our research here: https://t.co/wR8afQOlXa
@pmitu Early adopters. The breakthroughs people will look back on haven't happened yet.
World modeling, continual learning, truly grounded enterprise AI, these are still open research problems. At @skyfallai, that's exactly where we're focused. Exciting time to be building.