On GAIA, a benchmark for evaluating General AI Assistants on solving real-world problems, Manus has achieved new state-of-the-art (SOTA) performance across all three difficulty levels. Manus was evaluated using the same configuration as its production version for reproducibility.
OpenAI o3-mini is a good model, but DeepSeek r1 is similar performance, still cheaper, and reveals its reasoning.
Better models will come (can't wait for o3pro), but the "DeepSeek moment" is real. I think it will still be remembered 5 years from now as a pivotal event in tech history, due in-part to the geopolitical implications but for many other reasons too.
All this discussed in 5 hour technical podcast I just recorded on the state of AI industry. Out tomorrow (hopefully).
A great illustration of the first-mover effect: nothing happens before the first demo, then once the existential proof is established, everyone else catches up quickly against all odds.
Just a year ago, it was unthinkable that any other model would even remotely approach GPT's lead. Today, Sonnet-3.5 (not even Anthropic's biggest Opus model) is already slightly above and Llama-3-400B is around the corner.
Just 4 months ago, Sora blew everyone's mind and seemed so out of reach. Today, we have at least 4-5 clones of Sora at 70-80% quality, such as Kling, Luma, and Runway. The clones wouldn't have rallied without OpenAI's first move.
Many companies have the technical muscle. But very, very few have good taste in projects and strong will to execute. First movers take tremendous risk, yet their advantages don't stick for too long.
Congrats to @Tesla_Optimus team on another stellar update! The video gives us a peek at their human data collection farm, which I believe is Optimus' biggest lead. What does it take to build such a pipeline? Optimus nailed all of the following:
1. Optimus hands are among the best 5-finger, dexterous robot hands in the world. It's got tactile sensing, 11 degrees of freedom (DOF) compared to many competitors with only 6-7 DOF, and robustness to withstand lots of object interactions without constant maintenance.
2. Teleoperation software: we can see that the human operators are wearing VR goggles and gloves. It is very non-trivial to set up the software to have first-person video streamed in and precise control streamed out, while maintaining extremely low latency. Humans are highly sensitive to even the smallest delay between their own motions and the robot's. Optimus has a fluid whole body controller that enacts the human poses in real-time.
3. Sizeable fleet: you need more than one robot to collect data in parallel, well-trained human contractors taking multiple shifts per day (preferably 24/7), and an on-call maintenance crew to make sure that the robots are always busy. That's a ton of operational complexity that academic research labs don't even think of.
4. Tasks & environments: it's equally important to figure out *what* to teleoperate. Currently, most such efforts are demo-driven: collect data on the tasks that you want to put into a social media video. But solving general-purpose robots requires us to think carefully about the distribution of tasks and environments. From 43"-51" in the video, we can see factory & household settings like moving batteries, handling laundry, sorting daily objects into shelves.
It's an open-ended research question: if you only have the budget to collect training data for 1,000 tasks, what would you pick to maximize skill transfer and generalization?
Closing thought: teleoperation is a necessary but insufficient condition to solve humanoid robotics. It fundamentally does not scale. More about this later.
In South America I saw this too A LOT
Many hotels, shops and office buildings had remote security people and doormen
You can see them and they can see you
Not sure where they work from but might as well be the cheapest country in South America (or the world!)
Remote jobs going to the next level now