eitan shaar

Building… | ex-AI Researcher @ Mentee Robotics

[eitan shaar @EitanShaar]

Amazing work! High-quality simulation is a key ingredient for advancing robotics

[Jimmy Wang @jwang771]

We introduce 3D World Simulator, an interactive world model that runs at 50+ FPS on an RTX 5090.

Our model predicts in a unified 3D representation space, capable of generating:
• Novel view appearance
• Multi-view consistent RGB-D
• 3D geometry
• Spatially aligned tactile signals

Play with the world model now at https://t.co/eveClAxURp!

[eitan shaar @EitanShaar]

Cool work!! When building software becomes this cheap, we no longer have to accept the same product as everyone else. We can build tools around our own taste and needs. Some of the best AI products may start as one person fixing something that annoys them.

[Andrej Karpathy @karpathy]

Weekend hack: HackerTube. I was annoyed with YouTube's recommendations so I had Claude filter down 43M comments over 20 years of HN to 6,454 videos that keep being recommended. *Significantly* better viewing experience, ~2 hours with Opus 5.5:
https://t.co/JIhXN7OSh1

- Excellent recs without any logins (state is kept in local browser session)
- Thumbnail is a representative frame from middle of the video instead of the super duper A/B test hyper-optimized user uploads
- Underlying data from the comment analysis as one zip file, point your Claude at it and ask for your own version
- HackerTube Longs ✊

A bit more on the process:
I use Claude Code (desktop app for Mac), Opus 5.5 on Medium effort as my default. I only talk to Claude and have it do everything, the only "manual work" I helped with in tandem was in the browser (reminiscent of MenuGen): 1) I had to create some kind of API key from Google Cloud UI to query YouTube for video metadata, 2) I booted up a VPS box for serving and 3) I bought the domain name. Once the box was up I gave Claude the IP and told it update, secure, harden, deploy and optimize the code, and it did the whole thing.

Many awesome projects that would have normally taken weeks/months even can now be few-shotted for breakfast. What else are we not thinking of.

[eitan shaar @EitanShaar]

Cool work!! This is a huge step toward testing how robots behave in simulation before putting them in the real world. We need to use these environments to find failures early, while we can still learn from them safely.

[Pranjal Aggarwal @PranjalAggarw16]

Introducing Gym-Anything for Robotics: Scaling realistic robot simulations.

200 RL environments across 25 simulations, including liquids, cloth, molten metal, soil, and even sound, in economically valuable domains! Best models get 0%

And we show that frontier models can learn via ICL using different methods, and even significantly improve VLAs like π0.5!

See video examples (sound on)

🧵

[eitan shaar @EitanShaar]

As robot data collection gets easier, choosing what to record matters more. We need simulation to identify where another real demonstration would help most, so human effort goes toward gaps rather than more repetitions of familiar tasks.

[Amish Sethi @amish_sethi04]

Can collecting robot data be as easy as 𝗽𝗿𝗲𝘀𝘀𝗶𝗻𝗴 𝗿𝗲𝗰𝗼𝗿𝗱?

Introducing UMI-Live, a 𝗰𝗼𝗺𝗽𝗹𝗲𝘁𝗲𝗹𝘆 𝗼𝗽𝗲𝗻-𝘀𝗼𝘂𝗿𝗰𝗲 bimanual UMI we made that takes you from handheld demonstrations all the way to a trained robot policy.

It's built to be easy to use. Each gripper has its own camera, LiDAR, compute, and battery, so there are no base stations to mount and nothing to calibrate in the scene. You press one button and both hands start recording.

While you keep collecting, our pipeline processes finished episodes in parallel and turns them into LeRobot datasets, automatically uploaded on Huggingface. VLAs fine-tuned mostly on this data succeed in 𝟭𝟬/𝟭𝟬 (π0.5) and 𝟴/𝟭𝟬 (MolmoAct2) trials on a real bimanual task.

We're open-sourcing everything, from the hardware designs and build guides to the code and ~40 hours of data 🧵

[Lukas Ziegler @lukas_m_ziegler]

how can you not be obsessed about robotics, like fr

[eitan shaar @EitanShaar]

The more a robot can do, the harder it becomes to know where it will fail. We need simulation to actively search for those limits: unfamiliar tasks, changing conditions, and combinations we never thought to test.

[Bernt Bornich @BerntBornich]

Broadness and generality is what everyone worships as the key breakthrough that made LLMs impactful enough to change the course of humanity

Yet most people don't get humanoids and want specialized robots ...

NGL, surprises me every time

[Chen Tessler @ChenTessler]

Project page: https://t.co/G1CyVlB9Ko
Code and model: https://t.co/K7icUfh5eO

Co-led with Yifeng Jiang, alongside our fantastic collaborators @AlecSaysThings , @YiShi_333 , @UmarIqb , @GalChechik , and @xbpeng4 .

[eitan shaar @EitanShaar]

Every real-world robot failure should become a test we can run again. Reconstruct the scenario, reproduce the failure, and check each new model against it. A failure is much more valuable when the next model has to prove it learned from it.

[RoboPapers @RoboPapers]

Full episode dropping soon!

Geeking out with @c_voelcker, @ArpitBahety, and @zhiyuan_zhou_ on OopsieData, a community effort to collect and learn from real-world robot failures.

https://t.co/yrCuYapsYP

Co-hosted by @chris_j_paxton and @DJiafei

[eitan shaar @EitanShaar]

@adcock_brett is an inspiration. Real-world data is essential, but robotics needs a path to scaling with compute, not just human labor. Capture real scenarios, turn them into simulations, and use compute to explore what we couldn’t afford to collect by hand.

[Brett Adcock @adcock_brett]

To scale data to these levels, we had to build a startup inside of a startup

I’ve held a daily standup for 6 months; still doing it 🫠

[eitan shaar @EitanShaar]

Simulation gives us control over the experience a robot gets: different objects, layouts and interactions. Its biggest advantage may be training on situations that are too rare or expensive to collect on hardware.

[RoboPapers @RoboPapers]

Using simulation to teach robots would unblock the massive scale of robotics data that seems to be missing right now. The issue is usually sim-to-real transfer; MolmoBOT aims to solve this with sufficient diversity of simulation data, so that no real data is necessary for robot deployment. It has a few parts: MolmoBot-Engine produces diverse pick-and-place demonstrations across many different simulated environments; MolmoBot-Data is a dataset of 1.7 million expert trajectories for item manipulation and pick-and-place tasks; and they train a number of VLA policies which can achieve strong performance on real-world pick-and-place tasks with zero real robot data. @rosemhendrix, @SnehalJauhri, Abhay Deshpande, and Maya Guru join us to give us the details.

To learn more, watch RoboPapers episode 113 with @chris_j_paxton and @ruijie_sg!

[eitan shaar @EitanShaar]

The next generation of cyberattacks won’t just target software. They’ll target machines acting in the physical world.

Agree with @chris_j_paxton that running models locally on robots makes sense. But local inference doesn’t guarantee safety.

What happens when an attacker learns how to manipulate the model and bypass its guardrails?

That’s exactly what we’re demonstrating here, using a Physical AI model’s API to bypass its safety guardrails.

Physical AI needs a new approach to security.

[Chris Paxton @chris_j_paxton]

I am confident the WHOLE robotics stack wont run in the cloud -- just the parts that require a lot of parameters

[eitan shaar @EitanShaar]

If we’re entering the era of Robotics APIs, we need to start thinking about the security threats that come with them, just like we’ve seen with LLM APIs.

Except this time, the models act in the physical world.

[Chris Paxton @chris_j_paxton]

Its pretty clear that the age of cloud-based, API-based robotics is here; that people are unlikely to be running the best models locally; and that the same forced that drove LLM scaling are going to drive robotics as well

[eitan shaar @EitanShaar]

The next leap in robotics will come from learning through experience.

SGS method (@rosario_scaling) is making that possible, unlocking entirely new capabilities.

Great work @octi_zhang !

[Octi Zhang @octi_zhang]

(1/6) From @GoogleDeepmind’s AlphaGo to AlphaZero, we saw machines find moves that reached beyond what we’d imagined for them. No one taught them those strategies. They emerged from reinforcement learning: playing, failing, and trying again under a single binary task reward for success or failure. Inspired by them, we couldn’t help but wonder: what might a robot discover if we gave it that same freedom to learn?

That question led us to AlphaNIST. Starting tabula rasa, with that same binary reward plus only a simple control cost, a single policy solves all 20 @NIST assembly tasks at a 98.2% success rate in simulation. More surprising still, AlphaNIST can complete 10 different tasks in a row, receiving that reward only once, after all 10 are done.

Just as AlphaGo’s strategies surprised its creators, AlphaNIST discovered dexterous strategies we couldn’t have come up with ourselves.

Paper: https://t.co/HZBYgG7SWK
Website: https://t.co/VyUKiYFI2y

[eitan shaar @EitanShaar]

The next leap in machine intelligence will come from those who know how to leverage simulation

[Ilir Aliu @IlirAliu_]

"You need more data, you need more data..."
What if not?

What if you could do this completely in simulation and it works.

What if your robots could learn to thread nuts, mesh gears, and insert pegs with precision, and then do it for real, using nothing but a regular camera?

No real-world data collection, but more impressive:

without any human demonstrations!

That’s exactly what researchers from the University of Washington and NVIDIA just demonstrated.

Their robots mastered these precise assembly skills entirely in simulation and transferred them zero-shot to real hardware. The same training approach also works for legged robots crossing much harder terrain than before.

The breakthrough was a simple, smart change in what the robot practices during training.

And did I mention no custom rewards for every task?

By focusing practice on tasks that are neither too easy nor too hard, the system keeps improving even when scaled up massively.

Huge thanks to Octi Zhang (@octi_zhang) and the team for openly sharing this work, it’s a strong example of how smarter training can reduce the endless need for real-world data.

📌 Project website: https://t.co/wGuglGmvx7
Paper: https://t.co/CAfOrmyS0W

Follow along if you are interested in robotics and AI!

——

Weekly robotics and AI insights.
Subscribe free: https://t.co/9Nm01QUcw3

[eitan shaar @EitanShaar]

Feels like next-gen models are right around the corner :)

[Diogo Almeida @CompleteSkeptic]

After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI?

I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev

• 20-200x faster
• 40-400x cheaper (w/ output tokens free)
• Frontier composable intelligence optimized for decisions

AFAICT the shortest path to AI-based economic revolution

[eitan shaar @EitanShaar]

@ariel__shaulov Cool 🚀

[eitan shaar @EitanShaar]

Couldn’t agree more

[Anto Patrex (r/aw) @antopatrex1]

every robotics company obsesses over the robot. almost nobody is building the thing that actually keeps robots running.

the monitoring layer. predictive telemetry that tells you a robot is about to fail 6 hours before it does.

this will be a massive standalone business:

[eitan shaar @EitanShaar]

@chris_j_paxton Do you think the end-state is that the task itself becomes the interface, with zero human intervention?

[eitan shaar @EitanShaar]

@Thom_Wolf Nice :) I’m waiting for Microducks videos to take over my feed haha

[eitan shaar @EitanShaar]

Finally robots are leaving the lab !!
Real robots. Real tasks. Real customers.

This is where things get interesting.

[Dyna Robotics @DynaRobotics]

After so many demos, models, pilots, robots are still struggling to land real deployments with real customers. Until now.

Today we’re excited to share that our robots have successfully crossed the ROI threshold, and Din Tai Fung, one of the highest revenue per location restaurant chain in the US, is rolling out Dyna robots across its extensive restaurant network.

This brings our rollouts across hotels, logistics, data centers, and many other use cases to a fleet that reaches hundreds of robots by the first half of 2027. And we’re just getting started.

It’s been a wild year, and today we’re double clicking on the battlefield stories and sharing a few learnings about scaling robot deployments. We are just scratching the surface.

Read the full blog post: https://t.co/8QVEoQuzBY