Benjamin Tay

robotics x ai

[Benjamin Tay @bennytay07]

Excited to announce the release of my survey paper on egocentric data!

World action models depend on training data that record agent-environment causality. Therefore, we mapped the current landscape by data through the lens of causal completeness, distilling it into three key tiers.

This is followed by an analysis of the saturation of the data collection economy, and discusses the problem of 'scaling in the dark' -> there is a lack of proper benchmarking / taxonomy that measures how well a dataset actually improves model performance.

Full paper:
https://t.co/DO3qnVwuH4

[Jakabfi Károly (霸王最忠之臣子) @JakabfiK]

Didn’t know the Chinese hotel wifi password so I just punched in 88888888 and it worked lmao

[Hubert Thieblot @hthieblot]

How technical are you?

0 — Elizabeth Holmes
10 — Ilya Sutskever

[Perceptron AI @perceptroninc]

Today we're releasing Mk1.5: a new intelligence layer for embodied agents.

It flies drones, controls quadrupeds, powers smart glasses, tracks objects, searches the web, reasons visually, and dispatches its own sub-agents.

One model, no platform-specific retraining. 🧵

[Benjamin Tay @bennytay07]

choco pies may be the greatest snack of all time, i can eat like 10 of these at once

[carra @stargazzeerrrr4]

If I had generational wealth, I would definitely be doing some useless master's in New York

[Sam Padilla @theSamPadilla]

After 2+ years in the robotics data space, we are shutting @Eidon_AI down.

The thesis was right. But the business is brutally hard.

We close this chapter by open-sourcing everything we built and sharing lessons for anyone venturing into the space. https://t.co/pyL9IkjQPT

[Benjamin Tay @bennytay07]

@hughphan_ hugh

[5 @comeuptuff]

what a privilege it is to be exhausted by an ambition of your own choosing

[Benjamin Tay @bennytay07]

@issyadelaja thanks king

[Benjamin Tay @bennytay07]

Excited to announce the release of my survey paper on egocentric data!

World action models depend on training data that record agent-environment causality. Therefore, we mapped the current landscape by data through the lens of causal completeness, distilling it into three key tiers.

This is followed by an analysis of the saturation of the data collection economy, and discusses the problem of 'scaling in the dark' -> there is a lack of proper benchmarking / taxonomy that measures how well a dataset actually improves model performance.

Full paper:
https://t.co/DO3qnVwuH4

[Israel Adelaja @issyadelaja]

man if i hear another australian talk about "tall poppy syndrome" im gonna lose it. it's starting to feel like a victim mentality because we aussies talk about it so much. there are ambitious people in australia but u just need to find them. there are communities like this in sydney (and i'm sure in other places in aus) if you look properly.

[Kenneth Cassel @KennethCassel]

have talked to a bunch of robotics founders and they all are inundated with outbound from dozens of robotics dev tools / data collection / infrastructure startups constantly

at least 10x more people building sw tools for robots instead of robotics companies

[Benjamin Tay @bennytay07]

why i am bullish on the future of drones

from first principles, the fundamental advantage of the drone (referring to a quadcopter) is that you can occupy any open space in 3D.

this enables you to view things from any angle - optimal for dangerous inspection e.g. inside of ship hulls, nuclear reactors, etc

as the models + hardware improve we will also see the mainstream utilisation of drones with manipulation capabilities, making it easier to retrieve things in extremely difficult-to-reach areas.

but they are energetically inefficient when it comes to large scale transportation. that is why i do not believe in large-scale drone package delivery, trucks are a more efficient form factor.

the main benefit of a drone comes from the ability to inhabit any position in 3D space, which is why they are most optimal for precise, tricky spots - i.e. deployed either for inspection, or for very tricky manipulation tasks. but they should not be the backbone of the system itself, that belongs to other forms like robot arms, humanoids, wheeled robots...

[Benjamin Tay @bennytay07]

a hardware deployment company is the exact same -> u build a domain specific harness on the particular industry you're in

consider all the rules, regulations, company-specific requirements that go into the construction industry.

the goal is to capture all this messy data, which will be very very difficult at the start, and will require long hours spent analysing stuff at the jobsite but that is ur moat because you create a data flywheel that makes each following deployment even easier and easier.

the sooner u get started on this the more of an advantage u have, eventually u will have a massive moat over whatever industry you're in. then u can use ur capital to further take over the stack -> i.e. build ur own hardware / models like what Harvey AI / Waymo is doing.

(p.s. thanks @issyadelaja for giving me his insight)

[Jessy Huang @goodhartproof]

I was just at YC demo day yesterday. Besides hardware/physical things, everyone is just basically just building a domain-specific harness

[Benjamin Tay @bennytay07]

@thehanazhang @Scobleizer valid, how do you think we can achieve this? right now every deployment feels like a consulting project lol

[Benjamin Tay @bennytay07]

people in the robotics industry are spending too much time indoors

everyone is focused on 'selling shovels': (inference, sim testing, data collection etc) but who is actually going outside and making these robots work in real applications?

[Benjamin Tay @bennytay07]

@ziqinyuan *teleoperation

[Benjamin Tay @bennytay07]

humanoid robots are the MOST useful type of robot

mainstream view: "humanoids are shit because why would u try make a human when u can just make lots of small specialised robots that do each job perfectly??"

1) humanoids satisfy a psychological need over a functional need. yes i agree its more optimal to make a bunch of tiny robots for each task, but fundamentally we as humans are drawn to machines that look like us (go to any expo and see).

having a cute looking humanoid is fundamentally more comfortable and more easy on the human experience than having ur whole house controlled by some robot swarm like ur living in some kinda facotry

2) our cities and everything around us has been designed for the HUMAN body, so to create a generalist requires a human-like embodiement. society needs generalist robots, e.g. in aged care + customer service.

also some things do NOT require precise perfect optimisation, e.g. i do not want my laundry to be mathematically optimised to have the perfect stack + fold, because that is lowkey creepy as fuck.

[Benjamin Tay @bennytay07]

@ziqinyuan humanoids are largely still shit because the current models aren’t good enough yet. but once development in world models improves -> humanoids stop rely on teleportation -> long term vision everyone have a personal humanoid at home

[Benjamin Tay @bennytay07]

@RazaRafiq1990 cool tell me more

[Benjamin Tay @bennytay07]

@Scobleizer we don’t need frontier world models for most industrial tasks - ACT or Imitation Learning works fine