Bryan Lim

Building and deploying agents and robots @eastworlds_io @virtuals_io. Prev @autodesk ai lab @imperialcollege @MIT

[Bryan Lim @bryanlimwt]

A lot of emphasis in robotics and the physical AI space has been put on the model. Deploying robots in real businesses showed us it’s more than that.

What matters is how fast we can go from task specification to reliable and valuable deployment. we’ve spent the past few months @eastworlds_io focused on building the pieces required for reliable robot learning deployment: data collection and curation, training models, inference, hardware setups, and feeding what breaks back in. here are some tasks we got our robot to do pretty reliably! Still so much to work on and excited to share more soon!

I’m in Austin for CoRL! If you’re working on any part of this stack, I’d love to chat, compare notes or just catch-up!

[Bryan Lim @bryanlimwt]

@vitalisvos19 Congrats @vitalisvos19! Amazing team!

[Bryan Lim @bryanlimwt]

@Ed__Johns @creandum @usv Congrats Ed!!

[Bryan Lim @bryanlimwt]

Here’s me telling a robot what to do, all from my phone, remotely while on the move during my daily transit! moving atoms and accomplishing physical work at command, wherever you need is the promise of robotics

[Bryan Lim @bryanlimwt]

try out prompting a real-robot at our robot playground at https://t.co/O9eYRxm7hZ featuring Dream-1 by @dreamlabs_ai now with more models coming soon!

[Eastworlds @eastworlds_io]

Props to @dreamlabs_ai for being the first robotics lab to release a publicly accessible API!

To let more people try Dream-1, we’ve set up a robot in our lab for anyone to experiment with.

This is an entirely new environment that the Dream Labs team had no hand in configuring, an impressive show of zero-shot generalisation!

[Bryan Lim @bryanlimwt]

Congrats @youliangtan @NamraByTheSea and the Dream Labs team!

[Dream Labs @dreamlabs_ai]

Today, Dream Labs is coming out of stealth with Dream-1, a robot foundation model that controls bimanual robots it has never seen, on tasks it was never shown, in places it has never been, from $3/hour.

Dream API is live for everyone, today. 🧵

[Bryan Lim @bryanlimwt]

turns out "robot-use" works on humanoids too. with a good enough whole-body controller/tracker, we let Astra/Fable/coding-style agents control a full humanoid robot.

speed, cost, precision, dexterity are all still problems which get compounded on a humanoid platform from poorer tracking and having to manage balance, which results in more correction loops. but "zero-shot" robot control for open-ended pick-and-place tasks is a pretty nice "emergent" capability

[Eastworlds @eastworlds_io]

Eastworlds and @UnitreeRobotics have officially partnered to accelerate the deployment of embodied AI.

As Unitree’s official Data and Deployment Partner, we will combine our data infrastructure, deployment capabilities, and regional footprint with Unitree’s leading robotics platform.

A major milestone for Eastworlds, and an exciting step forward for robotics in Southeast Asia.

[Bryan Lim @bryanlimwt]

@eugene_teoh @eastworlds_io happy to show you around when you're in town!

[Bryan Lim @bryanlimwt]

Excited to share a little bit of what I’ve been up to recently. Earlier this year, I left Autodesk after an incredible time developing foundation models for AEC. I've gone back to working with robots and have been building @eastworlds_io with an amazing team!

Robotics is a notoriously unforgiving space. Slower feedback loops, hardware breakdowns, and the open-ended chaos of the physical world is painful. But that pain is what makes seeing a robot autonomously operate, recover and reliably work in the real world even more magical (even if it’s just picking up a bottle!).

Eastworlds is building at the intersection of real-world deployment, data, models, and hardware. In particular, we're interested in rapid fuss-free robot deployments and high quality robot data from those real-world deployments. This simple model trained to reliably pick up bottles demonstrates our evaluation setup and validation of the data collected on the deployment and data platform we have been working on. We're privileged to work with early deployment partners in Asia who are equally excited about robots and being part of building the future and pushing the frontier of general-purpose robots and AI models that operate in the physical world.

Looking forward to sharing more soon. If you’d like to chat or are interested in working together (esp. if you want real-world data or want to deploy and evaluate your robot models in the real-world on real use-cases), reach out!

[Eastworlds @eastworlds_io]

At Eastworlds, we validate our robot data before we sell it.

Here, a Unitree G1 autonomously and reliably picks up a bottle using a model trained on Eastworlds' data with just $200 of compute.

[Bryan Lim @bryanlimwt]

Massive effort from amazing collaborators @allardmaxime079 @MFlageat and @CULLYAntoine. Super thankful for the opportunity and privilege to play a part in this work!

3/3

[Bryan Lim @bryanlimwt]

Super happy to share that our work on robot adaptation is now out in @NatureComms!

Imagine a robot being paralysed in some areas from unknown failures like a faulty or damaged wheel or motor, operating and driving such a vehicle is near impossible.

🧵1/3

[Maxime Allard @allardmaxime079]

🤖Thrilled to share that robotics work from my PhD is out in @NatureComms 🎉
"Getting robots back on track by reconstituting control in unexpected situations with online learning"

With @MFlageat , @bryanlimwt , @CULLYAntoine @imperialcollege
🧵below

[Bryan Lim @bryanlimwt]

Turning right becomes moving straight, going straight means turning left. This work is our attempt at addressing this through rapid online learning. Regaining control of the robot again and operating and driving it as if nothing was wrong felt magical!

2/3

[Bryan Lim @bryanlimwt]

@CULLYAntoine @imperialcollege Congratulations! 🎉

[Bryan Lim @bryanlimwt]

@vitalisvos19 Congratulations Vitalis!

[Bryan Lim @bryanlimwt]

More information in this 🧵 too!

https://t.co/2wOSoxz6KL

[Bryan Lim @bryanlimwt]

💡⚙️ Ideas and inventions are rarely generated and created in isolation 🏝️
We explore using few/many-shot prompting of language models with quality-diverse examples provided from QD algorithms for solution generation for QD problems!

Work done with @MFlageat @CULLYAntoine

[Bryan Lim @bryanlimwt]

Had a great time at @ALifeConf this week presenting In-context Quality-Diversity! Feel free to reach out if any of this is interesting #ALIFE2024

[Hiroki Sayama @HirokiSayama]

Bryan Lim discusses LLMs as an in-context quality-diversity generator
#ALIFE2024 @ALifeConf

[Bryan Lim @bryanlimwt]

Check out more in the paper!
📝: https://t.co/QfCIQHDs4t

[Bryan Lim @bryanlimwt]

💡⚙️ Ideas and inventions are rarely generated and created in isolation 🏝️
We explore using few/many-shot prompting of language models with quality-diverse examples provided from QD algorithms for solution generation for QD problems!

Work done with @MFlageat @CULLYAntoine

[Bryan Lim @bryanlimwt]

Compared to other QD algorithms which use a few solutions from the archive for the next offspring (i.e. parents for a mutation/crossover or the init of an ES mean dist.), In-context QD makes greater use of the quality-diverse solutions in the archive for solution generation