@wenhaocha1@ShivaSujit These things don't make it into papers, but they are at the base of everyone's resultsπ«‘
Also, for Robotwin we found that environment seeds stable on a machine with a given GPU were not stable on a different machine and GPU. Would be interesting to see if robodojo has same issues
Found similar behaviours in Robotwin and Polaris a couple of months back (lots of pain with @ShivaSujit to find stable initial conditions for more rigorous evals for our paper)
We need to do better in terms of sim stability
in our new blog, we take a look at RoboDojo sim, which are base on NVIDIA's Isaac Sim and it is a good robo benchmark used in Fable 5.2 model card.
the video pretty much show what we found. the simulation is not even wrong. I'm now fully understand why robotics experts "hate" the past simulator.
but from the first principle, I do believe LLM powered agent could do much much better simulation with the minimal but necessary real-world calibration is the right way we build embodied intelligence. real-world are slow, expensive, hard to scale, hard to get labelled, but sim not. then we partially solve the data problem for robotic and we could scaling up the compute!
@fanf666@ishanxnagpal VLAs are not zero shot (they still work OOD, but with lower performance), and they will probably not be in our lifetime. They are not trained on the whole internet like LLMs afteral;. That's fine though, there is no reason to not do on-site data collection for robots
@ishanxnagpal Would be interesting to see also how much time is spent reasoning/tool calling vs actual env running, as latency is the main argument against this approach.
If you manage to better use the world knowledge of these models I agree it can definitely go further, so good luck!
@BasaniAshwath SGS https://t.co/EjG7tZHA3Q show that large scale RL in sim enables also zero shot contact rich real world capabilities! Molmobot https://t.co/KIhPbkMgfu instead does only SFT in sim and show zero shot to pick and place tasks only, so maybe is a matter of SFT vs RL
1/2
Couple of thoughts after #CoRL2026
- Zero-shot Sim2Real has come to manipulation: many teams are using it in place of IK, but it seems to work also at the policy "system 1" level once simulation is scaled enough
- Many World models applications are still not explored and
lots of talks on how people want to use them
- Agentic robotics intended as "Claude to low level actions" is useless (for now). CaP and trajectory generation have shown interesting applications (like the demo from microagi). To become more widespread and useful we need openmodels
itβs driving me INSANE how people are ignoring this footage is sped up 60x with pauses removed!
The capability jump is impressive but please STOP saying βrobot foundation model labs are dead !!!!β when CLEARLY these are unusable levels of inference speed and reliability
VLAs run 4β10 Euler steps per action. What if you just ran 1?
Across 6 VLAs, 4 sim benches and 4.8k real-robot rollouts, 1 step largely matches multi-step, while being 1.6-4.8Γ faster, w/o ANY retraining or distillation.
π #CoRL2026 #111, Nov 9th 5pm (@lucanunz presenting)
@YoussefMMSaied Really interesting work! In our CoRL paper (https://t.co/COjrp7t6OW) we also find that iterative flow integration doesn't always help in terms of task performance, but it does give more behavior diversity. Cool to see how a Gaussian head might help recover thatπ
Our 4800 A/B real world rollouts (around 22h) across 6 models are all open source! (This will also be part of https://t.co/TYKxcnYkSl π)
Take a look
- website: https://t.co/AXdX7CHoJe
- paper: https://t.co/MHQcowgCU0
- dataset: https://t.co/NxwGEx3MDv
See you at @corl_conf !
If you are using a flow matching VLAs, have you tried to do one step inference?
In our #CoRL2026 paper, we find that 1 Euler step often matches multi step inference for large, well-trained VLAs. No retraining or distillation. π§΅
Beyond success rate, we also investigated trajectory smoothness in the real world and diversity on multi modal benchmarks
- A jerk analysis reveals no harm to trajectory smoothness for larger models
- One step may lead to less diverse behaviours while even increasing performance
We evaluated 6 VLAs and one DiT model across multiple simulation benchmarks and 4800 real-world rollouts.
Our finding: extra inference steps often provide little benefit for larger models and performance are often equivalent.
@DominiqueCAPaul@tomprimozic Since you have annotations on a scale 1-5 for the quality of the episode, and also some speed measure (since you collected slower episodes on purpose), you could try inserting metadata in the prompt like pi0.7 paper and see where that gets you. The blog was very interesting btw!