Probably spent 5 weeks worth of GPT-6 Astra usage. It doesn't seem to me like it's generally smarter, or even better at long horizon tasks than the already pretty good 5.6 sol. I keep having to keep it on track and pointing out obvious things. It's much better at _specific_ things though, like blender, app use or images though.
Wonder if this is going to be a new trend, models will have more things they are really good at, but still not quite human-level at building the larger picture.
What problems and bottlenecks keep coming up in physical AI research?
We analyzed 64 episode transcripts of the brilliant @RoboPapers podcast to see which bottlenecks robotics researchers mention the most.
https://t.co/NTaG4WqlOf
In March, we mapped the pain across 64 RoboPapers episodes ... this time, we mapped the surprises that most changed our thinking. Here are eight of the surprises after scanning those same episodes.
TL;DR: better systems around a model often beat better models.
https://t.co/ou1yIzZxzh
When running robot policies, we established that cloud had to be at least twice as fast as edge before we felt it worthwhile (including all the network, etc. latency). Where is that crossover for your product?
@physical_int@nvidia Where does the world-model evaluator disagree most with real rollouts: contact-rich transitions, object state, or errors that compound over a long task?
@LeRobotHF@allen_ai We saw MolmoAct2 run smooth basic pick and place on SO-101, but it did not distinguish the colors or cube vs cylinder we tried. Does the zero-shot eval include appearance-based instructions, or mainly spatial pick and place?
MolmoAct2 (so101 ft) does well in some basic pick and place tasks and runs smoothly on @HapticLabsAI's inf. infra. A bit surprised that it's not been able to distinguish colors or shapes (e.g. doesn't distinguish between a yellow or a blue cube, or between a cube and a cylinder)
Robot training pipelines do not always fail loudly
We harvested 267 public datasets tagged 16-DOF OpenArm. Only 45 merged cleanly: 652 episodes and 583k frames
Read about how we mapped what broke and had an agent curate the corpus and fine-tune Pi0.5 ↓
https://t.co/ePKAhxgV05
Three open robot models dropped in three days. @LeRobotHF 0.6.0 on Tuesday, with six benchmarks to evaluate a policy before it touches hardware. LingBot-VLA 2.0 the same day: a 6B open checkpoint trained on 60,000 hours of real robot data. @Nvidia GR00T 1.7 this morning, on a 2B backbone.
Where @HapticLabsAI steps in: we do everything after the download. Shrinking a policy to one robot's compute budget, serving it at control-loop rates, and managing a fleet where some inference runs on the robot and some in the cloud.
IMO when it comes to physical AI, SV is a bit too frontier-pilled.
Don't get me wrong, frontier labs are VERY important to the ecosystem, but I think there are many problems in agriculture, construction, even consumer that can be solved with off-the-shelf hardware and don't need 99.999% success rates, and I suspect they are not as interesting to discuss so they are (accidentally) under-rated.
What we really need are entrepreneurs who UNDERSTAND the business case and pain point, and are willing to assemble the product, get costs down, and pound the pavement for distribution.
I meet many robotics companies that fit the bill, and I'd argue this is an underrated real bottleneck for robotics: people fighting the good fight.
“Fun” fact: when VCs reach out to talk to “get smart about the physical AI space”, I often elicit more interest when I mention some arXiv paper than the fascinating role of systems integrators in the industry.
Weirdly, the less technical the VC the MORE the arXiv or latest droid demo seems to engage.
I strongly suspect SV doesn’t appreciate their role in robotics and physical AI deployment in real world. A friend’s theory is:
A. “From a distance, system integrators look closer to electricians and plumbers 👩🔧👨🔧 than they look to Stanford research PhDs 🧑💻👩💻.”
B. “System integrators don’t raise monster rounds” 😛
This is why I always find my conversations with @avizurlo to be very enlightening. He goes deep into this important part of the ecosystem.
Pattern I keep seeing: physical AI startups with software/math/AI founders in SF, NYC, or Austin (eg VC-friendly areas) recruiting remote founding hardware engineers in the Midwest.
CMU is the OBVIOUS hub, but I've met teams with folks in Indiana, Illinois, Wisconsin, Michigan.
Tons of latent CNC/CAD/logistics talent there with no idea coastal startups are hunting for them.
Got a chance to see an early preview of this before launch and @mexitlan and the @HapticLabsAI team has done an amazing job!
Great to see more people contributing to open source robotics like this. The ecosystem needs a lot more efforts in this direction.
Excited to see where this goes.
At #FestivusAI, we’re building a catalog for robotics.
Our first milestone was simple but important: gather scattered robotics data from #HuggingFace, #GitHub, vendor sites, tinkerer blogs, and all the odd little corners of the internet where useful knowledge tends to live.
We credit sources clearly. No mystery meat data.
Then we brought everything into one searchable platform, so people can actually find what they need without opening 47 tabs and questioning their life choices.
Now we’ve added an AI agent that makes pages easier to edit, improve, and keep current.
Think Wikipedia for #robotics, but faster to contribute to, easier to maintain, and built for the people actually doing the work.
And yes, our code and data are #opensource too.
Why? Because that’s how innovation leapfrogs. People build faster when the map is shared.