Adam Fishman 🐟🤖🐠

Robotics @ OpenAI, CS PhD from UW, 🍕 Enthusiant, He/Him

[Adam Fishman 🐟🤖🐠 @fishbotics]

Come pursue your wildest ideas at OpenAI robotics! Our team is full-stack, high execution, and aiming to solve some of the biggest, most exciting technical problems of our era

[Sam Altman @sama]

OpenAI Robotics is hiring, looking for exceptional full-stack hardware, ops, systems, and ML engineers to help us program and manufacture robots that are useful for society.

AI should be able to help people in the physical world. In the short term, we are focused on robots to support skilled workers to build our future infrastructure; in the long term, we imagine everyone having a personal robot doing anything they need.

Our world simulation research program, led by Aditya Ramesh (@model_mechanic), has evolved over the past year into OpenAI Robotics. Progress is rapid, and based on a foundation of co-design between robotics hardware and ML research.

If you love working hands-on across the robotics stack and want to build the future, please consider joining us. Send an email with your background and evidence of exceptional accomplishment to: [email protected]

[Adam Fishman 🐟🤖🐠 @fishbotics]

Headed to ICLR next week! It's my first time at this conference. Anyone have tips?

[Adam Fishman 🐟🤖🐠 @fishbotics]

Super excited about these roles. If you (or anyone you know) would be a good fit, please reach out! We are a small, super collaborative team with big ambitions.

[Caitlin Kalinowski @kalinowski007]

Really excited to be posting our FIRST Robotics hardware roles for @OpenAI, including two very senior tech lead engineering (IC) roles and a TPM Manager.

The first role is for an **EE Sensing Engineer** to help us design the sensor suite for our robots.

The second role is for a **Robotics Mechanical Design Engineer** with experience designing gears, actuators, motors and linkages for robots.

The last role is for a **TPM Manager**. This will be a fun, scrappy role to start and will span Product TPM work, standing up our training lab, and keeping us running smoothly as we cycle through our product design phases.

I can't wait to get started on this work. I just hit my two month mark at OpenAI and can confirm the talent density, work velocity and focus make this a really wonderful place to do technical work. Join us!

(Links in comment)

[Adam Fishman 🐟🤖🐠 @fishbotics]

@micoolcho It would be very interesting to evaluate how a human would perform if they tele-operate a robot. I assume it would depend on the tele-operation interface

[Adam Fishman 🐟🤖🐠 @fishbotics]

🚀 Excited to present our latest research at @corl_conf in Munich! Meet "Avoid Everything" – a novel, model-free approach to collision avoidance that brings robots closer to safely navigating real-world, cluttered environments. 🤖🦾

🧵👇

[Adam Fishman 🐟🤖🐠 @fishbotics]

@micoolcho Ah good question. I suppose it depends on what kind of human expert you use. If it’s a person moving their own arm, I would guess their collision rate would be super small, maybe even zero. But, also humans can safely make contact because we are compliant and sense by touch

[Adam Fishman 🐟🤖🐠 @fishbotics]

7/n For those in Munich this week, come check out our poster Thursday morning (session 2). We’re thrilled to share this work with the robotics community and explore what’s next in safe, autonomous motion! 🌟

[Adam Fishman 🐟🤖🐠 @fishbotics]

6/n For those interested in the technical details, we’ve shared the paper on our project website and will open source the code soon (end of November): https://t.co/SBQX55Scrv 📂💻

[Adam Fishman 🐟🤖🐠 @fishbotics]

@viddivj Me too! Would love to catch up at the conference :)

[Adam Fishman 🐟🤖🐠 @fishbotics]

@kalinowski007 @OpenAI Super excited to work together and learn from you! Welcome!

[Adam Fishman 🐟🤖🐠 @fishbotics]

@dhruvpatel2012 Oops! I opened up my DMs 😅. I'll send you a message and we can find a time. BTW this paper looks awesome.

[Adam Fishman 🐟🤖🐠 @fishbotics]

I will be at CoRL in Munich this week! DM me if you want to grab a coffee and chat about big models for general robotic intelligence (or anything else) 🤖🧠

[Ria Doshi @riadoshi21]

🤔 Can we train one policy to control a wide range of robots, from drones to quadrupeds, navigators to bimanual manipulators, and more?

🦾Introducing CrossFormer: a single policy that can perform manipulation, navigation, aviation, and locomotion:

https://t.co/GOSCByd58Q

[Adam Fishman 🐟🤖🐠 @fishbotics]

This work was in collaboration with a bunch of amazing researchers at University of Michigan: @BoyangWang7, Nikhil Sridhar, Chao Feng, @MarkVanderMerwe, @NimaFazeli7, and @jjpark3D

[Adam Fishman 🐟🤖🐠 @fishbotics]

In contrast to other areas of ML, supervised robot learning is unable to leverage internet-scale data because we just don't have internet-scale demonstrations of robotic behavior. In our new paper, This&That, we sidestep this challenge by leveraging a diffusion video model (1/n)

[Boyang Wang @BoyangWang7]

Verry happy to share our new paper, This&That, an dynamic robot video generation model with language and simple gestures conditioning! Moreover, we also propose Diffusion Video to Action (DiVA) model to transfer generated videos to robot actions in the rollout environment.

[Adam Fishman 🐟🤖🐠 @fishbotics]

To leverage these gesture-and-language conditioned videos, we propose a novel architecture that we call DiVA. It consumes the generated videos alongside perceptual input (images) as conditioning and produces action chunks to execute on the robot (6/n)