ControlWiz

Building & thinking out loud

[ControlWiz @Control_wiz]

@Lavnir_ 😂

[ControlWiz @Control_wiz]

I have to stop arguing on here.

[Sparr @sparr0]

@Control_wiz What work? This wouldn't even warrant a publication. The implementation would just be things like "don't ever follow closely enough that them braking could cause a collision", "don't ever enter their turning radius", etc. all the worst case bounds for any more-informed simulation

[Deb Raji @rajiinio]

I do not want to do AI research that is reactive to what these companies are doing, or even what they're saying.

The entire field keeps chasing after product releases. Some spend more time reading marketing copy than their colleagues papers and I just... do not want to do that?

[ControlWiz @Control_wiz]

@sparr0 Never heard of such work, and highly doubt it exist. Share references.

[ControlWiz @Control_wiz]

In multi-agent planning you react and replan continuously using your belief states at 30–50 Hz. If the behavior models of the other agents are known, many methods exist. When they are unknown, you need some predictive model, and collision avoidance still cannot be guaranteed without strong assumptions. I’m not aware of a general method that guarantees safe behavior without these, please share if you do.

[Sparr @sparr0]

@Control_wiz @ChenTessler No, you don't need some assumption about how other agents will act. It is possible to drive such that you always have an escape route. To never follow so closely that you can't stop or evade if they brake. To never cross ahead of someone that might speed up.

[Fei-Fei Li @drfeifei]

Scientific research is fundamental to advancing civilization and helping people globally to solve the most critical problems, from medicine to materials, from brain science to physics, and much beyond. This is only possible when scientists have access to the best tools of the time to conduct scientific research, including having access to AI-based tools.

[ControlWiz @Control_wiz]

@ChenTessler AVs also perform worse with bad weather , road debris, and mixed traffic. In normal driving conditions, they’re often better than the average human driver.

[ControlWiz @Control_wiz]

Idk if human behavior is solvable. No car will likely reach level 5 autonomy, as long as there are human drivers.

[alex p. @alexpdiggs]

@Control_wiz just a lil speed bump.

[ControlWiz @Control_wiz]

@ChenTessler I agree. Never claimed AVs are not better than humans. My point was we don't have full autonomy because of humans on the road.

[ControlWiz @Control_wiz]

@diakou What article? I am not following.

[ControlWiz @Control_wiz]

One of my goals with this account is to share my honest thoughts on where I think the frontier of robotics actually is. I might sound like a pessimist, but I think the field deserves realism.

I’m happy robotics is getting all this attention, but I also have to admit that a lot of what we see in both industry and academia is exaggerated. Curated demos make the field seem much more advanced than it really is.

I’ll share my thoughts, for what they’re worth.

[ControlWiz @Control_wiz]

@daphne_cor @ChenTessler I agree. They are more safe than human drivers. My initial point was having fully autonomous vehicles on the road, and human behavior being the gap to lvl 5.

[ControlWiz @Control_wiz]

@cspaliwa1 You don't need dexterity for pick and place. You need it for a lot of home and industrial automation.

[ControlWiz @Control_wiz]

You need at least three contact points for dexterous manipulation. This was proven around thirty years ago. However, three finger grippers have many more DoFs and are often more than 10× expensive. Dexterous manipulation is also a very challenging control problem, at least classically.

[Kushal @kushalk_]

@Control_wiz What are your thoughts on 2-fingered grippers? So more dofs than parallel jaw grippers but still lacking the third finger.

[ControlWiz @Control_wiz]

@ChenTessler I agree. The bar for level 5 autonomy is unreasonable. Which is why I said likely impossible. It's not just an autonomy but a legislation problem as well.

[ControlWiz @Control_wiz]

Robots react much faster than humans. The challenge isn’t reaction time, it’s modeling unpredictable human behavior. In multi-agent collision avoidance, you need some assumption about how other agents will act. It’s difficult to make quantifiable assumptions about human drivers, especially in edge cases. Humans, through experience, quickly develop an intuitive model of the drivers around them.

[ControlWiz @Control_wiz]

World models are not data sources.

[Michelle @michellelsun]

What is the best data for training humanoid & robotics foundation models?

Pete Florence @peteflorence (CEO @Generalist, ex-Google DeepMind) dropped his live data tier list in this 7-minute clip on @tbpn:

- S-tier: Real-world robot experience (especially glove/sensor high-dexterity data)
- A/B-tier: Internet/YouTube videos. Surprisingly powerful for transfer learning (the “web data” moment for physical AI)
- B-tier: Text/common crawl (Reddit, books, etc.). Useful priors, but not enough alone
- C-tier: Motion Capture. Great for whole-body motion, weak on finger dexterity
- C or lower: Simulation / synthetic / world models. High potential, still waiting for strong real-world proof

Generalist has collected 270,000+ hours of real-world manipulation data (scaling ~10k hours/week). And Pete stressed one key point:

“The quality of data is incredibly important.”

It’s not just about volume. It’s developing intuition for what actually drives performance through hands-on work.

As Physical AI scales, curated real-world and high-quality internet video looks like a winning combo.

h/t @yuji_fujima

[ControlWiz @Control_wiz]

@ChenTessler I don't get your point about being unique to autonomy. We were discussing why we don't have fully autonomous cars.

[ControlWiz @Control_wiz]

You can only optimize if you have some model of the environment. Otherwise, you need a gazillion examples to learn it implicitly.

[Chris Paxton @chris_j_paxton]

I think one of the cool things about world models is just how much more efficient they seem to be in both data and compute.

[ControlWiz @Control_wiz]

@chetan_ Agree. When it's bimanual the second hand plays the role of the third contact point. My point was dexterous manipulation. Changing the object pose relative to the gripper without relying on another arm or the environment.

[ControlWiz @Control_wiz]

It's well known that you need three fingers for stable manipulation. With two fingers you are limited to pick and place tasks. The missing part has been tactile sensing. There are decent ones but they are very expensive. The gap is robust multi finger manipulation with cheap force sensors.

[Andile @FlexAppeall]

@Control_wiz What is your take on the current state of robotic grippers

[kache @yacineMTB]

Jokes aside, using frontier models to change the direction of human thought secretly sets an incredibly dangerous precedent - the fact that they were okay with setting this precedent speaks volumes about how extremist they actually are