Chelsea Finn

Asst Prof of CS & EE @Stanford
Co-founder of Physical Intelligence @physical_int
PhD from @Berkeley_EECS, EECS BS from @MIT

[Chelsea Finn @chelseabfinn]

LLM post-training used to mean fine-tuning to a downstream task

Robotics has been stuck in this setting, needing task-specific fine-tuning for best performance

π07 changes this: It works out of the box & outperforms fine-tuned specialists

Details: https://t.co/QbO3E4D3QN

[Chelsea Finn @chelseabfinn]

@ZeffMax We started at someone’s house in Saratoga when I joined!

[Chelsea Finn @chelseabfinn]

@OriolVinyalsML @JeffDean @Google @GoogleDeepMind Congrats Oriol!!

[Chelsea Finn @chelseabfinn]

@JeffDean @Sanjay_Ghemawat @OriolVinyalsML @quocleix @DiscoLoopAI Congrats Jeff!!

[Chelsea Finn @chelseabfinn]

I'm giving a talk tomorrow at ICML on emergent physical generalization, including π0.7 🤖

3:15 pm @ SCALE workshop in Ballroom 201
https://t.co/xppCOkktOJ

[Chelsea Finn @chelseabfinn]

I'm giving a talk on how we can move beyond the scalar reward bottleneck for both robotics & LLMs.

ICML RLxF workshop tomorrow at 1:30 pm.

[Shao-Hua Sun @shaohua0116]

RL is hitting a ceiling with human feedback. What if the world itself becomes the signal?

Join us at the RLxF: RL from World Feedback 🌍 workshop at ICML 2026 @icmlconf tomorrow (July 10th)!

Web page: https://t.co/cN0itnL1yI

[Chelsea Finn @chelseabfinn]

Project led by @marceltornev, @anubhamahajan01, @AbhijnyaBhat

Paper: https://t.co/2dyFJpxwU7
Code & videos: https://t.co/KBAPSbr6a8

Check out Marcel’s thread for more details!
https://t.co/tlUBH8VJqY

[Marcel Torné @marceltornev]

We should stop optimizing robot policies against a single overall reward. Trajectories differ along many axes, such as speed, precision, and subtask completion, and one can be better on some while worse on others. If we collapse all of that into a single overall axis we lose this structure making the reward ambiguous and harder to optimize.

Blog: https://t.co/WXWue03RVq
Paper: https://t.co/AvJ904Xt9S

[Chelsea Finn @chelseabfinn]

Freeform preference learning has multiple nice properties:

(1) It works better

When controlling for the number of preference queries, learning with multi-axis preferences yields far more performant policies than single-axis rewards.

[Chelsea Finn @chelseabfinn]

(3) Long-horizon credit assignment
Most robot RL focuses on short horizon tasks b/c dense temporal rewards are hard to get.

Freeform preferences yield dense rewards for subtasks without subtask segmentation.

[Chelsea Finn @chelseabfinn]

With freeform preference data, we train a lang-conditioned reward that captures all axes of a task.

We then train a policy conditioned on each reward axis and the corresponding reward.

FPL allows the robot to maximally leverage and learn from each axis of supervision.

[Chelsea Finn @chelseabfinn]

Freeform preferences let the supervisor define relevant axes and then specify preferences along those axes.

Axes can be either a fixed rubric or freeform language.

This eliminates ambiguity, allows for thorough coverage of all axes, and provides more dense supervision.

[Chelsea Finn @chelseabfinn]

Sparse rewards, progress metrics, and preferences are popular, but they
- often neglect many aspects of a task
- collapse many axes into one measure
- frequently yield ambiguity and disagreement across annotators

We instead propose freeform preference learning

[Chelsea Finn @chelseabfinn]

Long term, we need reward models to capture all aspects of performance, like success, outcome quality, and speed.

This also includes task-specific axes like:
- was the PB spread evenly?
- was the apple slightly bruised while bagging it?
- was the furniture bumped or scratched?

[Chelsea Finn @chelseabfinn]

We’ve been approaching reward supervision for robots the wrong way.

I think freeform preferences are part of the answer.

A short 🧵

[Chelsea Finn @chelseabfinn]

LLM RL optimizes for sequential reasoning

We also optimize over the reasoning strategy, incl parallel trains of thought, aggregation of parallel traces, & sequential reasoning

This allows the model to better explore & allocate compute at test time

https://t.co/DkTSllkmvp

[Jubayer Ibn Hamid @jubayer_hamid]

The most capable reasoning systems in AI scale inference compute along several axes: sequential compute to think longer, parallel compute to sample many independent attempts, and aggregative compute to synthesize prior traces into a new improved one. But during training, we only optimize how models use sequential compute. This creates a fundamental mismatch between how we ultimately deploy these systems and how we train them, leaving much of search and synthesis unoptimized.

We introduce SPIRAL, an RL framework for making all inference-compute primitives end-to-end learnable: models learn to coordinate sequential, parallel, and aggregative reasoning using only the reward of the final output. Work with @ifdita_hasan (co-lead), @michaelyli_ , @oshaikh13 , @yoonholeee , @DorsaSadigh , @chelseabfinn , @noahdgoodman 🧵

[Chelsea Finn @chelseabfinn]

Can we translate a rough sense for what to do + VLA prior into successful behavior?

Flow reversal steering:
1) runs the VLA's flow ODE backwards to back out noise that's closest to coarse traj
2) runs flow forwards to get closest good behavior

Paper: https://t.co/FUDqDEpeJq

[Andy Tang @tangerinecoder]

Generalist robot policies learn many useful skills, but struggle to select good behaviors for new tasks. To solve this, we introduce Flow Reversal Steering (FRS), a method to refine coarse semantic guidance into precise, in-distribution motions.
https://t.co/uCR6KmoDo8

1/N

[Chelsea Finn @chelseabfinn]

How does test-time scaling impact robots?

We find that larger models, more thinking, and more context help significantly for some prompts but not others.

Like LLMs, we can also train a router to for a better performance/latency tradeoff!

Paper: https://t.co/HEjjCkrsen

[Jadelynn @_jadelynn]

test-time compute [ttc] in robotics isn't free & isn't always worth it. smart allocation of ttc recovers frontier-level planning at a fraction of the cost!

coauthor @milanganai
w/ Yasmina @ajaysridhar0 Mozghan @katielulula Clark Barrett @jiajunwu_cs @chelseabfinn @drmapavone 🧵

[Chelsea Finn @chelseabfinn]

Can robot foundation models collaborate with themself?

We finetune a VLA to be able to control any robot in a team.

- matches or outperforms training separate models or a single centralized model for all robots
- readily scales to large teams

Paper: https://t.co/yqdSOQ7ead

[Ria Doshi @riadoshi21]

🤔 Can we train one VLA policy to control multi-robot teams without any explicit communication?

✨ Introducing CHORUS: a single policy for decentralized, multi-embodiment collaboration

🧵⬇️

[Ji Woong Kim @jwbkim]

We show that robots can learn high-level task semantics, such as sorting rules, skill composition, and rule-based ordering, directly from human demos.

This is useful because if your target task is a composition of the robot's existing skills, you could just collect human demos for it without collecting further robot data.

Introducing Ego-Pi: VLA fine-tuning for egocentric human and robot data, a collaboration between @Stanford and @Meta.

Website: https://t.co/dIF6n4QGy3
Paper: https://t.co/3GFk6KQw9P

1/6

[Chelsea Finn @chelseabfinn]

Scaling RL to long horizons remains a major challenge.

Long-horizon Q-learning (LQL) prevents compounding bootstrapping errors by bounding the difference in value over long horizons.

It shows large gains over 1-step TD and n-step returns!

Paper: https://t.co/OTk3M6cz8p

[Armaan Abraham @armaanabraham]

In RL, what if we could learn from any experience from any policy in a way that is reliable and scalable?

This would be helpful in domains like robotics where new data is expensive.

We introduce Long-horizon Q-learning (LQL) to tackle this https://t.co/1Ckb5ZePyo.