Pedro Morgado

Building robot foundation models @SkildAI
prev Professor @UWMadison · Postdoc @CMU · PhD @UCSD

[Skild AI @SkildAI]

We just hit 100M ARR within 10 months of starting deployments.

We are in factory lines. On construction sites. In kitchens. In data centers.

Cleaning. Welding. Building. Cooking.

Deploying.

[Deepak Pathak @deepakpathak]

One surprising aspect of S1’s in-context learning (ICL) is where it shines most: super long-horizon tasks (10+ min) and scenarios outside the pretraining distribution.

For short, simple pick-and-place tasks (5–20 sec), most frontier models can already perform well either zero-shot with no prompt or from just a language prompt. With enough pretraining data, the model often already “knows” what needs to be done. @DrJimFan also highlighted this distinction eloquently here: https://t.co/vbZWh3NmTy

Once you move to genuinely unseen scenarios and long-horizon tasks, in-context learning becomes critical. This shows up dramatically in the scaling laws: S1’s ICL advantage over regular VLAs grows exponentially as we move further out of distribution.

We discuss this in detail in the S1 blog:
https://t.co/xpUtioZM5K

[Brian Zhan @brianzhan1]

Skild, Generalist, and Sunday published their charts inside five weeks, allowing us to finally measure the progress of robotics foundation models objectively.

GPT-3 mattered because in-context learning showed up: put examples in the prompt, get tasks the model never trained on.

Sunday's ACT-2 folds laundry at 99.1% in homes it's never seen. But it's one task the lab picked, and the curve is squeezing against its ceiling. It proves scaling perfects for only a chosen skill.

Generalist's GEN-1.5: loss falling through 8 months of training, and 59% one-shot success on 10 novel tasks. These results are impressive. But the loss chart tracks prediction error, not task success. And 59% is a single measurement at a single scale. One point can't show you whether scaling is working.

Skild tests their robotics model on the most accurate benchmark. In-context learning claims only mean something relative to how far the eval sits from training. They put that distance on the exam, and scale on the x-axis.

By that benchmark, the Skild S1 stands alone. On unseen tasks, one video prompt, zero fine-tuning, they are at ~0% at 1k hours. 7% at 10k. 25% at 30k. 66% at 100k. Convex, still bending up.

In-context learning should be graded against task unseen and is it long horizon. S1 clears both, using ten minute tasks with dozens of steps that never appear in training.

Skild will be sharing more about how they achieved these results soon.👀

[Deepak Pathak @deepakpathak]

We’re scaling S1 as fast as we can. The curve shows no signs of slowing down, steadily going up.

We’ve been working for a while to find this kind of scaling law in robotics.

[Jiachen Zhu @JiachenAI]

Excited to see S1 finally go public! 🚀 This release also marks exactly one year since I transitioned from vision research to robotics. If I had to share the single biggest lesson from that journey, it’s this:

Before asking "How do we train a better model?", we need to ask "How should the model behave at inference time?"

In academia, we tend to attack problems the same way: set up a benchmark, tweak the architecture or training recipe, and watch the score climb. Today, much of robotics research runs this similar loop: pit VLAs against WAMs, introduce a new loss function, test a different action representation. Model performance on benchmarks keeps improving, but inference-time behavior stays simplistic and fixed.

But my past year at @SkildAI taught me that raw model performance isn't the ultimate driver of real-world success. Inference-time behavior is.

The real world guarantees high stochasticity and endless out-of-distribution scenarios. The dominant deployment paradigm (taking the past X seconds of observations to predict the next Y seconds of actions) won't get us to general-purpose robotics, because every robotic action has compounding consequences. A vision model can guess wrong on a static test set with zero impact on the next sample. A robot cannot. It operates in a closed loop: its own mistakes create the very distribution shifts it must then survive. "Collect more data, fine-tune, redeploy" is not a scalable answer. To truly generalize, a model must adapt on the fly.

So the order of operations has to invert. First, define an inference-time behavior scalable and robust enough to handle the real world. Then work backward to shape data collection and model training. Everything in the S1 release stems from this inversion.

Language modeling already taught us this lesson: LLMs didn't crack complex multi-step math just because someone threw more tokens at pre-training. Instead, the breakthrough came from changing the inference-time behavior first (reasoning step by step) and then redesigning training around it. Betting that data scale alone will magically produce emergent physical skills is just as naive. Robotics needs the same shift.

S1 is proof that working backward from real-world inference changes everything. And this is just the starting point.

Check out the release below, and enjoy the rest of your day! 👇

[Deepak Pathak @deepakpathak]

This is a good question. If the task was only 5–10s long, it can be difficult to distinguish true physical steerability from generalization. But we’re doing tasks that run for over 10 minutes and not seen during training (OOD). Sustaining performance at such long horizon and novel tasks would be impossible without physical steerability actually happening. Hope this clarifies.

[Pedro Morgado @PedroMorgadoX]

Learning to make pancakes. From a single demonstration. Unseen before. How cool is that?

[Skild AI @SkildAI]

Introducing S1, our new foundation model that learns from one example.

It can be taught 10-minute long tasks that it has never seen before, from one video prompt without any fine-tuning.

Watch S1 operate in real-time via in-context learning: