HardCoreAI

@HardCoreAI

Research Scientist @LumaLabsAI || Prev Senior Research Scientist @Bytedance Seed Multimodal

Sunnyvale, CA

Joined November 2016

71 Following

17 Followers

21 Posts

HardCoreAI retweeted

amit

@gravicle

3 months ago

We are launching Open Physical AI Lab at Luma to solve generalization in robotics We believe general physical AI is the most consequential problem to solve to improve life on Earth. Allowing one or very few companies to control the entire intelligence and physical infrastructure of the world is a profound threat to the world. This is the path LLMs took, and it is the path physical AI is on today. To solve this problem of generalization in physical AI and to maximally benefit all of humanity, we are establishing the Open Science Physical AI Lab (OPAL Lab) at Luma. Today, physical AI and robotics are in the pre-AI-scale era. Unlike LLMs and multimodal models which are capable of solving a wide range of previously seen and unseen problems, robots are trained on and can only replay specific tasks in specific environments from small scale examples, limiting their utility. Physical AI has a data gap, creating a generalization crisis. The default solution is to simply scale teleoperation to capture data across every possible task and combination of tasks – a physically and economically impossible approach. Over the past 4 years, through fundamental research on multimodal AI, Luma has trained frontier foundation models in 3D, image, video, and unified language-visual generation. In this process, we have built vast infrastructure to make sense of raw internet-scale multimodal data and train efficient generative models with it. Building these systems has given us unique insights into how raw multimodal data can be used to learn physics, perception, reasoning and interaction with the physical world. Over the past few months, mounting empirical evidence from our work on Unified Models has turned this realization into a strong scientific conviction. Generalization is not unique to text. The Open Physical AI Lab will research and scale methods to solve generalization and develop World Models for understanding and interaction with the physical world. This generalization will lead to robot systems proliferating everywhere - factories, research labs, hospitals, streets, and even our homes. Given the societal and economic upside, this transition will be faster than any other technology in history and these systems will be embedded into every layer of human life. Physical AI will help us solve labor shortage and stagnation plaguing world economies. Physical AI will become the means of production in every nation. Allowing one or very few companies to control the entire intelligence and physical infrastructure of the world is a profound threat to humanity and democratic freedom. This is the path LLMs took, and it is the path physical AI is on today. To reverse this trajectory, this lab will be an open science effort. We will use our scale and expertise to build the substrate and make it available so that any dedicated group of people can use it, modify it, and assemble it into systems of productive work. To accelerate progress, we will collaborate with our peer labs and academia on core research, build evaluations to measure progress and assess safety, and partner with industry leaders on chips, hardware, and physical agent systems. If you share this conviction and believe that this foundation should be built in the open for the benefit of all humanity, please come work with us.

gravicle's tweet photo. We are launching Open Physical AI Lab at Luma to solve generalization in robotics

We believe general physical AI is the most consequential problem to solve to improve life on Earth. Allowing one or very few companies to control the entire intelligence and physical infrastructure of the world is a profound threat to the world. This is the path LLMs took, and it is the path physical AI is on today. To solve this problem of generalization in physical AI and to maximally benefit all of humanity, we are establishing the Open Science Physical AI Lab (OPAL Lab) at Luma.

Today, physical AI and robotics are in the pre-AI-scale era. Unlike LLMs and multimodal models which are capable of solving a wide range of previously seen and unseen problems, robots are trained on and can only replay specific tasks in specific environments from small scale examples, limiting their utility. Physical AI has a data gap, creating a generalization crisis. The default solution is to simply scale teleoperation to capture data across every possible task and combination of tasks – a physically and economically impossible approach.

Over the past 4 years, through fundamental research on multimodal AI, Luma has trained frontier foundation models in 3D, image, video, and unified language-visual generation. In this process, we have built vast infrastructure to make sense of raw internet-scale multimodal data and train efficient generative models with it. Building these systems has given us unique insights into how raw multimodal data can be used to learn physics, perception, reasoning and interaction with the physical world. Over the past few months, mounting empirical evidence from our work on Unified Models has turned this realization into a strong scientific conviction. Generalization is not unique to text.

The Open Physical AI Lab will research and scale methods to solve generalization and develop World Models for understanding and interaction with the physical world. This generalization will lead to robot systems proliferating everywhere - factories, research labs, hospitals, streets, and even our homes. Given the societal and economic upside, this transition will be faster than any other technology in history and these systems will be embedded into every layer of human life. Physical AI will help us solve labor shortage and stagnation plaguing world economies. Physical AI will become the means of production in every nation.

Allowing one or very few companies to control the entire intelligence and physical infrastructure of the world is a profound threat to humanity and democratic freedom. This is the path LLMs took, and it is the path physical AI is on today.

To reverse this trajectory, this lab will be an open science effort. We will use our scale and expertise to build the substrate and make it available so that any dedicated group of people can use it, modify it, and assemble it into systems of productive work. To accelerate progress, we will collaborate with our peer labs and academia on core research, build evaluations to measure progress and assess safety, and partner with industry leaders on chips, hardware, and physical agent systems.

If you share this conviction and believe that this foundation should be built in the open for the benefit of all humanity, please come work with us.

674

327

49K

HardCoreAI

@HardCoreAI

Last Seen Users on Sotwe

Trends for you

Most Popular Users