Sonny George

PhD Student @ Pitt

[Sonny George @s_george_iv]

@paolodiprodi @AgneyaTharun On my benchmark, Gemini Argon and GPT-6.1-Sol (full thinking) are good pairwise discriminators... W/ these models as discriminators plus maybe “agentic diverse robo-demo generation,” I think you could create a massive “style”-signal dataset, but not at my current budget hahah

[Sonny George @s_george_iv]

@paolodiprodi @AgneyaTharun I feel like for discriminating what behaviors correlate w what “styles” (lang modifiers), the problem is data. Whether a vid-capable Qwen or a lang-conditioned WM learning the “stylistic” priors for (fast) discrimination, IMO, the (open) data doesn’t exist at "general" scale yet.

[Sonny George @s_george_iv]

@paolodiprodi @AgneyaTharun Not sure I follow. I.e., Action(s)->WM => "score of how in-distribution/human-like”? Would the WM be generative from language such that you could prompt, e.g., “a humanoid walks **with swagger**” then use the lang-conditioned distribution of actions to score for such “styles”?

[Sonny George @s_george_iv]

@AgneyaTharun @paolodiprodi However, they seem to be good at random emotional things like downward head tilt -> something being done more "glumly". But overall it's a mixed bag, and the worst category was modification by analogy, e.g., wipe "like a dj scratching a record."

[Sonny George @s_george_iv]

@AgneyaTharun @paolodiprodi CVPR submission is a video understanding benchmark of "execution style modifiers." Qwens struggle w/ unexpected things like not being able to score a soft drawer closing as being done more "gently" than an obviously violent drawer slamming.

[Sonny George @s_george_iv]

@AgneyaTharun @paolodiprodi Only just starting RL exps... The only obviously positive example from my few tries is a G1 pick+placing w/ a "while keeping the other arm still" Qwen3.5-9b reward every 2s. Not very useful since that's easy to program a reward for.

[Sonny George @s_george_iv]

@AgneyaTharun @paolodiprodi Also trying task-agnostic “style” video-language joint representation learning too. E.g., for querying a dataset for stylistic anomalies. Happy to share ideas, thoughts, or even collab.

[Sam Earle @Smearle_RH]

The academic "we" finally always makes sense.

[Ken Goldberg @Ken_Goldberg]

If you're at #IROS26, completing your PhD soon, and interested in a Postdoc position to collaborate with me and my students on Agentic Robotics for bimanual manipulation in the AUTOLab @UCBerkeley, please come talk with us at the Bistro near the Westin Hotel Registration Desk between 7-8pm tonight! We're particularly interested in PhDs with skills in model-based methods, control theory, computer vision, and motion planning who are excited to work with frontier LLM and VLM models!

[Sonny George @s_george_iv]

@RaresAmbrus @WaldenRobotics The morphology looks awesome! Mega congrats!

[Sonny George @s_george_iv]

Excited to see these guys emerge from stealth mode!

[Rareș Ambruș @RaresAmbrus]

I'm incredibly excited to share what we've been building: @WaldenRobotics - a full-stack, human-centric, customer-focused robotics company on a mission to build general-purpose robots and deploy them in the real world.

The road here has been extraordinary. We've built the entire stack and pushed our robots into production. The hardest problems in AI and robotics aren't research problems anymore: since February, our robots have been doing useful work alongside people at a Toyota factory in North America. And most importantly, they learn and improve every day on the job. Here's a glimpse of our work so far: https://t.co/OTX7nT7kME

It's the privilege of a lifetime to be building this alongside my co-founders: @RussTedrake, @Ben_Burchfiel, @adnothing, Kerri Fetzer-Borelli, Siyuan Feng and Dave Johnson, and a phenomenal founding team.

What lies ahead is even more ambitious. If you're driven to get robots doing useful work in the real world, join us.

[Sonny George @s_george_iv]

@pham_blnh Very aesthetically pleasing! Someday I hope somebody makes a browser wrapper around MuJoCo teleop (client teleop->websocket to MuJoCo->simulation on remote server->updated scene to client viz). It wouldn’t be latency-optimal, but it would be fun.

[Sonny George @s_george_iv]

@roeiherzig Thanks for sharing!

[Sonny George @s_george_iv]

@JieWang_ZJUI @DJiafei @GRASPlab 1. Pick and place w/ objects that are described (not named); e.g., “pick up the long, yellow thing”

2. Action modification (e.g., “do X swiftly” = faster than “do X”?)

[Sonny George @s_george_iv]

@DominiqueCAPaul As long as it is AI-powered. Can't let your investors know you're using a rule-based coffee-machine.

[Sonny George @s_george_iv]

@ryan_punamiya I have thought about this a lot. I _think_ the most functional/useful framing is not linguistic or conceptual, but merely “whatever action/scope we are confident the low-level controller can handle” = subtask, which is situation-by-situation.

[Sonny George @s_george_iv]

@shirazkn @JesseMMichel Thanks for the reply! I think I did misunderstand; “cruising” is >.8 from the velo distribution across many episodes (not only the current episode, which a teleoperator could easily “troll”). Obvious in hindsight. Thanks!

[Sonny George @s_george_iv]

@AntonioSitongLi Right, _the servos_. To me, servos and grippers are different things. You mentioned using the _gripper_’s deformation as signal. That’s what I’m curious about. Are u saying the Feetech servo’s driver also tells you how the gripper is deformed? Thx! 🤘❤️

[Sonny George @s_george_iv]

@JesseMMichel I noticed y’all used 80th velo percentile as the proxy for “prolly moving through free space” (“cruising”). How’d you arrive at that number specifically? Knowing this, could a teleop theoretically “troll” the system by moving fastest during the precision-needing moments?