Ge Yan

CS PhD Student @UW | Previously @UCSanDiego | Intern @ToyotaResearch @allen_ai

[Ge Yan @GeYan_21]

Introduce ManiFlow 🤖, a visual imitation learning policy for general robot manipulation that is efficient, robust, and generalizable:

- 98.3% improvement on 8 real-world tasks, generalizing to novel objects & backgrounds
- Applied to diverse embodiments: single-arm, bimanual & humanoid robots with increasing dexterity
- Unified policy works with 2D RGB images or 3D point clouds
- Generate precise high-dimensional actions in just 1–2 inference steps
- Trained from scratch, yet achieves superior efficiency and scalability compared to the large-scale pre-trained π0 model

Web: https://t.co/tVKyyamVHo
Open-sourced code: https://t.co/nOuGeaT0Be

[Ge Yan @GeYan_21]

@imankitgoyal Congrats!

[Nicklas Hansen @ncklashansen]

Super excited to share the last paper of my PhD: "Hallucination in World Models is Predictable and Preventable"✨

We train a 350M-param generative world model on a large dataset w/ 210 tasks and show that we can predict *when* hallucination happens and use that to fix it!

🧵1/n

[Hansheng Chen @HanshengCh]

This might be some good evidence that V-JEPA-like "world models" are nothing special compared to a generic generative model. They are all dumb, only that some appear less dumb at scale while others don't scale efficiently.

[Jitendra MALIK @JitendraMalikCV]

We have seen some impressive robot manipulation policies recently. This is great, but for these results to be convincing (and practical) we should insist on generalization across 1. Object location (within some range) 2. Different instances of an object category 3. Background clutter. Authors should present experiments which demonstrate the range of variation which can be handled. Far too often the policy doesn't even generalize across all instances of an object category! Legged locomotion policies were convincing only when they worked across different terrain as in https://t.co/APzhehXfPT We need to do the same for manipulanda.

[Richard Sutton @RichardSSutton]

RL Ethics has a Predictive Semantics

I would like to try to explain the view of ethics and values that arises from my research in reinforcement learning in simple, layman’s terms that are accessible to all.

Reinforcement learning agents seek to maximize their reward over time, where reward is essentially pleasure minus pain. This is not quite hedonism, because the maximization takes into account all the consequences, long-term as well as short. A reinforcement learning agent might endure pain to get a larger pleasure later, or forego an immediate pleasure if it stored up later, greater pain. Formally, reward is a number at each time step, and the reinforcement learning agent seeks to maximize value—the sum of the rewards at future time steps. (This could be defined precisely with some math.)

The assignment of rewards to time steps is a free choice that defines the agent’s goal; different agents could have different rewards, and there is no basis (yet) for preferring one set of rewards over another. Value though is a different matter. Given a world and a way of generating rewards, the true values at each time step are fully determined. The rewards are primary, dependent on nothing else, whereas the values are secondary, following from the rewards (and the dynamics of the environment). In decision making, the agent should make the choice that leads to highest immediate value, not highest immediate reward.

If rewards are arbitrary, values follow from the rewards, and correct behavior follows from the values, then all seems straightforward. What about all the complexities and controversies of ethics? Some of these are still present, arising because the values, though well defined, are initially unknown and can be difficult to calculate or learn.

If the agent has knowledge of the world, then it may be able to calculate the values, but to do so exactly generally requires too much knowledge, computation, and memory. In practice, in new situations the calculation must be done partly at decide time, and cannot be done to completion without slowing down action selection too much.

In the absence of knowledge and computation, but given a generous allocation of memory and time, the agent can alternatively learn the values, again approximately. It is common for the agent to store an approximation to the world’s state’s values, and then to gradually improve these approximations—these predictions of subsequent rewards—by further experience. The stored approximate values are immediately available estimates of the desirability of situations; they are directly analogous to our intuitive sense of good and bad. They are ready for immediate use, but may only be rough approximations to the true values. They may be made more accurate with calculation (if the agent has knowledge) or learning (with more experience).

This completes the explication of the value system of the individual. Next we will go on to consider the value systems of groups. But the individual forms such an essential foundation that is never replaced, so let’s dwell on it a moment longer by reviewing its stark tenants: Each agent wants to get pleasure (reward) from the world. Pleasure is built-in to the agent and obvious when it happens, but when it will happen depends on the world and must be learned or calculated—and the world is too complex for either of these methods to yield answers that are completely correct. That is, every state of the world has a real, objective value (the amount of pleasure that will follow it), but estimates of its value are subjective. Forming better value estimates is a major cognitive task. They are a key intermediate step towards getting more pleasure from the world. Agents work on this all the time. It determines what they do.

If an agent lived alone, then this would be the end of our discussion of values and ethics. But people are not solo agents. Peoples’ worlds are comprised, in part, of other people, and this has many impacts of their attempts to estimate value and obtain reward. They live within groups of agents with whom they interact frequently and whom are major determinants of their success is obtaining reward. And thus, to achieve our reward, each of us must take into account, as best we are able, the rewards and values of those around us.
…

The most important insight is that it's alright, and perhaps obligatory, for the ultimate value to be hedonic (based on reward), as long as it is not "selfish" (disregarding the impact on others). The ultimate meaning of something being good, or right, or ethical, or moral, is that it will probably have a good outcome for the individual. Whether it will or not is extraordinarily difficult to calculate, so instead we use heuristics—approximations using features of a situation. The mistake is to think that those features are definitional rather that approximate predictive. The real definitional meaning of good is that it turns our well for us on average.

[Jiafei Duan @DJiafei]

4 years have been simply amazing! I’m happy to share that I have successfully defended my PhD!

Thank you to everyone who came to support me, and most importantly, to my thesis committee, advisors, collaborators, friends, and family for being part of this journey.

[Xiaolong Wang @xiaolonw]

Excited to share that Assured Robot Intelligence (ARI) has joined @Meta to help build the future of humanoid intelligence!

When we started ARI one year ago, our mission was clear: achieve physical AGI. Through deep customer engagements and real-world deployments, it became clear to us that serving the massive opportunity ahead requires training a truly general-purpose physical agent.

We believe this agent will be humanoid — and that scaling will come from learning directly from human experience, not teleoperation alone. Meta’s ecosystem brings together the key components needed to make this vision possible. We will be joining Meta Superintelligence Labs (MSL) to help bring personal superintelligence into the physical world.

We are incredibly grateful to the brilliant minds, robotics researchers, engineers, partners, and supporters who have worked with us on this journey. Thank you to our investors and angels, led by @aixventureshq , for believing in our mission.

This is just the beginning.

[Lerrel Pinto @LerrelPinto]

ARI is joining @Meta!

Over the past year, we have been building ARI (Assured Robot Intelligence) with the mission to build industry-grade physical AI for humanoids. The ARI stack is built on human experience, condensed into actionable tokens that can be rapidly adapted to real-world hardware.

But the most rewarding part of ARI has been the people. I feel truly blessed to have worked alongside some of the world's best roboticists, a top-notch investor pool led by @aixventureshq, and the many supporters pushing for us behind the scenes.

Starting next week, ARI will join the Meta Superintelligence Labs (MSL) to continue advancing frontier robotics models that advance personal superintelligence in the physical world. We have the potential to transform AI that can think and talk to AI that can do, assisting humans safely and reliably in the physical world.

To the many people behind the scenes who supported us: Thank you! This is just the beginning.

More in the Bloomberg article:

[Katherine Liu @robo_kat]

A few interesting rollouts from the Foundry-QwenVLA-2.5B multi-task model on seen tasks in sim –  a 🧵. I really like behaviors that involve non-prehensile manipulation, like the little nudges in StoreCerealBoxUnderShelf.

[Jean Mercat @MercatJean]

Releasing VLA Foundry: an open-source framework that unifies LLM, VLM, and VLA training in a single codebase. End-to-end control from language pretraining to action-expert fine-tuning — no more stitching together incompatible repos.

[Yi Ru (Helen) Wang @YiruHelenWang]

🤖How do we evaluate robots?
Fixed benchmarks.
Same tasks.
Predefined success.
‼️But this doesn’t tell us what robots actually understand.
➡️We built **RoboPlayground**.
✨TLDR: Evaluation should be a space anyone can define, not a fixed benchmark.✨

🧵1/n

#robotics #ai #robot

[Ge Yan @GeYan_21]

Try it out! Great hardware for anyone who wants to do tactile sensing.

[Binghao Huang @binghao_huang]

🤲Tactile sensing is powerful for robot manipulation, but hardware is still difficult to access, reproduce, and scale.
🎯That’s why we built FlexiTac: an open-source, low-cost, and scalable tactile sensing solution designed for real robotic systems.
• Project page: https://t.co/i3TJxfkHOD
We hope FlexiTac can help democratize tactile sensing for robotics research. (1/n)

[Sergey Zakharov @ZakharovSergeyN]

Our 3D Vision team (3DGR) is releasing Raiden — a data collection toolkit for YAM robots.

Built for scalable, high-quality data: supports leader–follower + SpaceMouse teleop, multi-camera setups, and modern stereo depth (incl. TRI learned stereo).

https://t.co/I4vXvVPuC4

[InSpatio｜影溯 @InSpatio_AI]

Spatial Intelligence just went Open Source. 🌐

Introducing InSpatio-WorldFM — the world’s first open-source real-time interactive 3D world model.

Last October, @theworldlabs's RTFM showed the vision. Today, we’re bringing that idea into the open for everyone.

✅Efficient, multi-view consistent spatial reasoning
✅Real-time interactive exploration
✅Open-source weights & code

Stop watching demos. Start building now. 👇

[Jiafei Duan @DJiafei]

With both TOPReward and RoboMeter now released, I’m sure people will compare them , and honestly, they’re both strong, just optimized for different goals.

For TOPReward, we’re pitching it as a general-purpose reward model that works across robotics and non-robotics. We go from an off-the-shelf VLM like Qwen-VL 3 (which, notably, wasn’t trained on robotics data) to TOPReward with no reward training, no fine-tuning, and no in-context prompting. And have fun building excited to see what people make!

But if you’re curious about a simple side-by-side between RoboMeter and TOPReward (implemented by @VilleKuosmanen), check it out 👇

[Marius Memmel @memmelma]

There’s a discussion going on rn about two recent robotic reward models: TOPReward⛰️ and Robometer🌡️

Which one is better? It depends entirely on your objective!

Here is a deep dive into the conceptual differences, strengths, and weaknesses of both. 🧵👇

[Jesse Zhang @Jesse_Y_Zhang]

A reward model that works, zero-shot, across robots, tasks, and scenes?

Introducing Robometer: Scaling general-purpose robotic reward models with 1M+ trajectories.

Enables zero-shot: online/offline/model-based RL, data retrieval + IL, automatic failure detection, and more!

🧵 (1/12)

[Elon Musk @elonmusk]

Anthropic is guilty of stealing training data at massive scale and has had to pay multi-billion dollar settlements for their theft. This is just a fact.

[Entong Su @EntongSu]

Pretrained diffusion/flow policies are powerful — but brittle at deployment.

We introduce RFS, a data-efficient RL framework that:
• steers latent noise for global adaptation
• applies residual actions for precise local correction

Works in sim and real-world dexterous manipulation 🖐️🤖

👉📄 Paper + videos: https://t.co/HumWkk7MdL

[Hao Zhang @HaoZhang623]

As video world models become increasingly powerful, do we still need explicit 3D?

A commonly misunderstood point is this: video world models are not “just 2D.” Their ability to maintain multi-view consistency, temporal stability, and realistic interaction necessarily implies that their latent knowledge encodes 3D world structure. Without some notion of 3D, consistency itself would not be possible.

The real distinction, therefore, is not whether a model has 3D but whether that 3D exists implicitly or explicitly.

Implicit 3D lives inside latent spaces and network weights. It supports generation, but it is difficult to localize, edit, constrain, or reason about.
It allows the world to exist, but not to be used.

Explicit 3D, in contrast, exists as structure and state: it is addressable, editable, composable, and transferable.
Its purpose is not better visual fidelity, but operability to allow the world to be manipulated, controlled, and executed.

From this perspective, video and 3D are not competing paradigms but a layered system:

2D/video is the interface to human perception; 3D is the interface to the physical world.

They can reinforce each other, but neither forms a closed loop on its own.

In practice, data not model architecture sets the upper bound of world models.
Explicit 3D may not be the final user-facing representation, but it is likely the most effective pathway toward scalable, high-quality, and controllable data.

Through explicit 3D/4D representations, worlds can be constructed systematically:
interactions can be programmatically sampled, states and actions can be composed, rendered into images and videos, and fed back to train video world models.

Seen this way, 3D is not the destination it is the starting point for scaling.

What truly drives progress forward is never the model itself.
Whether we capture the world or imagine new ones,
whether data comes from observation or intent,
whether we model what is or what should be the direction of the world is ultimately determined by human choice and purpose.

Models may extend the world,
but humans decide where it goes.
#Genie3 #worldmodel