i'm obsessed with what's happening in AI reforestation right now
this Franco-Brazilian startup called MORFO took a patch of land in Brazil that was rock-hard and compacted from years of cattle farming. they replanted it using a single drone. months later the ground was covered in grass, bushes, and small trees. the land came back to life.
here's how the whole thing works.
1. drones scan the terrain with high-resolution cameras and sensors
2. AI analyzes the imagery alongside soil samples, moisture levels, slope, and surrounding vegetation
3. the system picks from a catalog of 300+ native species, deciding exactly which plants will thrive in which specific spot
4. the drone fires biodegradable seed pods packed with seeds, nutrients, and moisture at 180 capsules per minute
5. satellite and drone imagery monitors regrowth over time, with AI tracking vegetation cover and biodiversity
6. two people and one drone cover 50 hectares a day. a person planting by hand manages about one hectare.
and MORFO isn't alone. AirSeed in Australia drops 250,000 seed pods per day into bushfire-scarred koala habitat, replanting swamp mahogany that koalas depend on to survive. Flash Forest in Canada fires 50,000 pods daily into wildfire-destroyed boreal forest, planning the replanting alongside Cree Indigenous communities. re-green won Prince William's Earthshot Prize after planting 6 million seedlings across 30,000 hectares of Amazon and Atlantic Forest.
five companies across four continents built this same approach independently. nobody coordinated. the physics of the problem demanded it.
knowing which seeds belong in which soil used to require years of ecological fieldwork, manual planting crews, and budgets that made large-scale restoration nearly impossible. now two people with a drone and an AI model trained on local soil data can replant 50 hectares before lunch.
this is the AI work that'll still matter in 50 years.
๐ฏ๐ต Japanese company Kawasaki has unveiled CORLEO, an innovative robotic horse powered by a hydrogen engine and equipped with artificial intelligence.
It constantly analyses the position of both the robot and the rider, ensuring that the rider is securely held in the saddle.
@SaltyGoat17@LeadingReport Not true. Leading Report is an American website and Twitter account that describes itself as a "leading source for breaking news". It is known for promoting misinformation and conspiracy theories about COVID-19and United States politics.
- per https://t.co/w02v5EuuTG
@AmericaPapaBear@LeadingReport Not true. Leading Report is an American website and Twitter account that describes itself as a "leading source for breaking news". It is known for promoting misinformation and conspiracy theories about COVID-19and United States politics.
- per https://t.co/w02v5EuuTG
@LetsTalkReal23@LeadingReport Leading Report is an American website and Twitter account that describes itself as a "leading source for breaking news". It is known for promoting misinformation and conspiracy theories about COVID-19and United States politics.
- per https://t.co/w02v5EuuTG
@Bubblebathgirl@LeadingReport Leading Report is an American website and Twitter account that describes itself as a "leading source for breaking news". It is known for promoting misinformation and conspiracy theories about COVID-19and United States politics.
- per https://t.co/w02v5EuuTG
YO! I just donated to the Surfrider Foundation to protect what I love, our ocean, waves, and beaches. Please consider joining me with your support today! https://t.co/FHfGdyioA4
ChatGPT can now see, hear, and speak. Rolling out over next two weeks, Plus users will be able to have voice conversations with ChatGPT (iOS & Android) and to include images in conversations (all platforms).
https://t.co/uNZjgbR5Bm
Let's reverse engineer the phenomenal Tesla Optimus. No insider info, just my own analysis. Long read:
1. The smooth hand movements are almost certainly trained by imitation learning ("behavior cloning") from human operators. The alternative is reinforcement learning in simulation, but that typically leads to jittery motion and unnatural hand poses.
There're at least 4 ways to collect human demonstrations:
(1) A custom-built teleoperation system - I believe this is the most likely means used by Tesla team. Open-source example: ALOHA, a low-cost bimanual robot arm and teleoperation system by Stanford AI Labs (https://t.co/8iXpiHVEjS). It enables very precise, dexterous motions, such as putting AAA batteries into a remote or manipulating contact lens.
(2) Motion Capture (MoCap): apply the MoCap systems used for Hollywood movies to capture the fine-grained motions of hand joints. Optimus' 5-finger hand is a great design decision that enables a direct mapping - there is no "embodiment gap" from human operators.
For instance, a demonstrator can wear a CyberGlove (https://t.co/S8hxErsEuU) and grasp the cubes on the table (as shown in video). CyberGlove will capture the motion signals & haptic feedback in real-time, which can be re-targeted onto Optimus.
(3) Wearing gloves & markers can be clumsy. An alternative way to do MoCap is through computer vision. DexPilot from NVIDIA enables marker-less and glove-free data collection. The human operator simply uses their bare hands to perform the tasks. 4 Intel RealSense depth cameras and 2 NVIDIA Titan XP GPUs (yeah, 2019 work) translate the pixels to precise motion signals for robot learning.
(4) VR Headset: turn the training room into a VR game, and let humans "role play" Optimus. Use the native VR controller or CyberGlove to control the virtual Optimus hands. This has the advantage of scalable remote data collection - annotators from around the world can contribute without coming onsite.
VR demonstration technique appeared in research projects like the iGibson home robot simulator, an initiative that I participated in at Stanford: https://t.co/eyI4ORkH6G
Above 4 are not mutually exclusive. Optimus could use a combo of them for different pros & cons.
2. Neural Architecture. Optimus is trained end-to-end: videos in, actions out. I'm quite sure it's implemented by a multimodal Transformer with the following components:
(1) Image: some variant of efficient ViT, or simply an old ResNet/EfficientNet backbone (https://t.co/L6PLTQJnGA). The block pick-and-place demo doesn't require sophisticated vision. The spatial feature map from the image backbone can be tokenized easily.
(2) Video: two ways. Either flatten the video into a sequence of images and produce tokens independently, or have a video-level tokenizer. There're numerous ways to efficiently process video pixel volumes. You don't necessarily need Transformer backbones, e.g. SlowFast Network (https://t.co/qDdXzqwJQp) and RubiksNet (https://t.co/CQU8D7TZgx, my paper at ECCV 2020, efficient CUDA shift primitives).
(3) Language: it's not clear if Optimus is language prompted. If it is, there needs to be a way to "fuse" the language representations into perception. FiLM is a very lightweight neural network module that serves this purpose (https://t.co/VI4TpgQ22V).
You can think of it intuitively as a "cross attention" of language embedding into the image-processing neural pathway.
(4) Action tokenization: Optimus needs to convert the continuous motion signals into discrete tokens for the autoregressive Transformer to work. A few ways:
- Directly bin the continuous values for each hand joint control. [0, 0.01) -> token #0, [0.01, 0.02) -> token #1, etc. This is straightforward but could be inefficient due to the long sequence length.
- The joint movements are highly dependent on each other, which means they occupy a low-dimensional "state space". Apply VQVAE to the motion data to obtain a shorter-length, compressed token set.
(5) Putting the above pieces together, we have a Transformer controller that consumes video tokens (optionally with language modulation), and outputs action tokens, one step at a time. The next frame from the table is fed back to the Transformer, so it knows the consequence of its action. That gives the self-corrective ability shown in the demo.
I believe the architecture is most similar to:
- Google RT-1: https://t.co/dpuon1bqU6
- NVIDIA VIMA: https://t.co/Tn3L63uGrv
3. Lastly, I'm genuinely impressed by the hardware quality. The motions are fluid, and the aesthetics is amazing as well. As I mentioned above, it's such a great decision to follow human morphology closely, so that there is no gap in imitating humans.
Atlas from Boston Dynamics only has simple gripper-style hands. In the long run, Optimus' bi-dexterous, 5-finger hands will prove far superior in daily tasks.
Congrats to @Tesla_Optimus team & @elonmusk ๐! I look forward to seeing the bots roam Mars some day ๐ฆพ
Thanks Little Hands for spending 118 minutes with me in 2022. I couldnโt stop listening to Acka Backa Soda Cracker. #SpotifyWrapped https://t.co/Fke4JueX0Y
The Tripitaka Koreana - carved on 81258 woodblocks in the 13th century - is the most successful large data transfer over time yet achieved by humankind. 52 million characters of information, transmitted over nearly 8 centuries with zero data loss - an unequalled achievement. 1/
OMG. This past weekend, 220 people in the US were killed by Gun Violence. The current 7-day moving average of new COVID 19 deaths in the US is 255. That's about the same. We need action on this.
https://t.co/wkY6ZwA0sw