In-Context Learning Results Hint at a “GPT Moment” for Robotics
For general-purpose robots to be useful and economical, you need to be able to teach them new skills on the fly. And now we can see how this will work, with in-context long horizon video demonstrations shown by companies like Generalist, Skild, Rhoda, Rhoda, and RobbyAnt.
The idea is simple: prompt with a demonstration, mixed video and proprioception information. Language isn't enough -- it doesn't specify the problem well enough. And it's starting to really work.
more in blog post
@EzMhypeL @Shiraz_AI Great question! You can think of the example as a prompt that we provide to the robot. So yes a human will then evaluate the robot behavior given the prompt (example), and you can do this instantly becuase we are not retraining / fine-tuning the model.
Everyone in robotics is waiting for the ChatGPT moment but … missing the most important part of the ChatGPT playbook.
So we left Boston Dynamics and Amazon Robotics to fix that, and start @Shiraz_AI to build robots that learn on the job.
When GPT-3 was released, it couldn’t do every task out of the box. But you could show it a few examples in the prompt, and it adapted on the spot.
We’re bringing that to robotics.
Show our robot a task once, and it adapts on the spot.
That’s exactly what our manufacturing customers need, especially when production lines change every week and traditional automation can’t keep up. And we’re already deploying it with our first customers.
Have a factory that’s difficult to automate? We’d love to talk.
@_k_sridhar@Shiraz_AI Thanks @_k_sridhar! Yes totally, we think in-context learning will open new markets and will also have important implications on steerability.
@hamudinaanaa@theworldlabs I would say an easy pipeline that can go from videos of our lab setup, to a simulation that respects the visuals and physics of the workspace.
@mario_bollini Thanks Mario!! Yes totally, being able to adapt to a new workflow is a necessary condition we see for our customers and for delivering value at scale.
Everyone in robotics is waiting for the ChatGPT moment but … missing the most important part of the ChatGPT playbook.
So we left Boston Dynamics and Amazon Robotics to fix that, and start @Shiraz_AI to build robots that learn on the job.
When GPT-3 was released, it couldn’t do every task out of the box. But you could show it a few examples in the prompt, and it adapted on the spot.
We’re bringing that to robotics.
Show our robot a task once, and it adapts on the spot.
That’s exactly what our manufacturing customers need, especially when production lines change every week and traditional automation can’t keep up. And we’re already deploying it with our first customers.
Have a factory that’s difficult to automate? We’d love to talk.
Everyone in robotics is waiting for the ChatGPT moment but … missing the most important part of the ChatGPT playbook.
So we left Boston Dynamics and Amazon Robotics to fix that, and start @Shiraz_AI to build robots that learn on the job.
When GPT-3 was released, it couldn’t do every task out of the box. But you could show it a few examples in the prompt, and it adapted on the spot.
We’re bringing that to robotics.
Show our robot a task once, and it adapts on the spot.
That’s exactly what our manufacturing customers need, especially when production lines change every week and traditional automation can’t keep up. And we’re already deploying it with our first customers.
Have a factory that’s difficult to automate? We’d love to talk.
Everyone in robotics is waiting for the ChatGPT moment but … missing the most important part of the ChatGPT playbook.
So we left Boston Dynamics and Amazon Robotics to fix that, and start @Shiraz_AI to build robots that learn on the job.
When GPT-3 was released, it couldn’t do every task out of the box. But you could show it a few examples in the prompt, and it adapted on the spot.
We’re bringing that to robotics.
Show our robot a task once, and it adapts on the spot.
That’s exactly what our manufacturing customers need, especially when production lines change every week and traditional automation can’t keep up. And we’re already deploying it with our first customers.
Have a factory that’s difficult to automate? We’d love to talk.
Everyone in robotics is waiting for the ChatGPT moment but … missing the most important part of the ChatGPT playbook.
So we left Boston Dynamics and Amazon Robotics to fix that, and start @Shiraz_AI to build robots that learn on the job.
When GPT-3 was released, it couldn’t do every task out of the box. But you could show it a few examples in the prompt, and it adapted on the spot.
We’re bringing that to robotics.
Show our robot a task once, and it adapts on the spot.
That’s exactly what our manufacturing customers need, especially when production lines change every week and traditional automation can’t keep up. And we’re already deploying it with our first customers.
Have a factory that’s difficult to automate? We’d love to talk.
Everyone in robotics is waiting for the ChatGPT moment but … missing the most important part of the ChatGPT playbook.
So we left Boston Dynamics and Amazon Robotics to fix that, and start @Shiraz_AI to build robots that learn on the job.
When GPT-3 was released, it couldn’t do every task out of the box. But you could show it a few examples in the prompt, and it adapted on the spot.
We’re bringing that to robotics.
Show our robot a task once, and it adapts on the spot.
That’s exactly what our manufacturing customers need, especially when production lines change every week and traditional automation can’t keep up. And we’re already deploying it with our first customers.
Have a factory that’s difficult to automate? We’d love to talk.
Everyone in robotics is waiting for the ChatGPT moment but … missing the most important part of the ChatGPT playbook.
So we left Boston Dynamics and Amazon Robotics to fix that, and start @Shiraz_AI to build robots that learn on the job.
When GPT-3 was released, it couldn’t do every task out of the box. But you could show it a few examples in the prompt, and it adapted on the spot.
We’re bringing that to robotics.
Show our robot a task once, and it adapts on the spot.
That’s exactly what our manufacturing customers need, especially when production lines change every week and traditional automation can’t keep up. And we’re already deploying it with our first customers.
Have a factory that’s difficult to automate? We’d love to talk.