Day 22/365 of robot learning.
Today I finally decided to start reading LoRA, a paper I've been putting off for a while but think is pretty foundational, especially since I want to contribute to pretraining as robotics foundation models continue to scale.
LoRA was introduced in 2021 as LLMs began to scale as well, with models like GPT-3 reaching 175B parameters, making full fine-tuning computationally expensive. It builds on the broader idea of parameter-efficient fine-tuning (PEFT), where we adapt models without updating all their pretrained weights. In their GPT-3 experiments, LoRA reduced trainable parameters by up to 10,000x and GPU memory requirements by 3x compared to full fine-tuning!
The main idea I grasped so far is to freeze the pretrained weights Φ₀ and learn a much smaller set of parameters Θ to represent the updates:
Φ = Φ₀ + ΔΦ(Θ), where |Θ| ≪ |Φ₀|.
More specifically, LoRA represents the weight update through two smaller matrices:
W = W₀ + BA
Where B ∈ ℝ^(d×r) and A ∈ ℝ^(r×k), with r ≪ min(d,k).
Instead of optimizing d×k parameters, we're only optimizing r(d+k), based on the paper's hypothesis that weight updates during adaptation have a low intrinsic rank, meaning they can be represented using much smaller matrices.
I've been slumped with projects and hw, so I plan to take this weekend to work through this paper in the background, but the robot learning grind doesn't stop!
Day 21/365 of robot learning.
I finally finished reading OpenVLA today! Unlike ACT or Diffusion Policy, the paper focused less on developing new algorithms and more on scaling existing architectures, improving data mixtures, and optimizing training, so it was pretty easy to get through.
It almost felt like a better application of the Bitter Lesson to robotics, borrowing a lot of the approaches that made LLM scaling successful.
I also couldn't help reading bits of π0.7 after watching Sergey Levine's recent conversation with Aditya Agarwal about zero-shot generalization and adapting to new tasks through simple language instructions rather than task specific fine-tuning, which OpenVLA relied on for new setups.
Here are a few key takeaways and thoughts on OpenVLA:
1. Scaling robot learning
OpenVLA was pretrained on 970K robot demonstrations, carefully filtering and reweighting datasets based on task diversity and performance. Surprisingly, they trained for 27 epochs, unlike the 1-2 epochs typically used for LLM/VLM pretraining.
It feels like scaling laws are starting to kick in, but we're still bottlenecked by how much human demonstration data we can collect. This made me wonder whether we even have enough global GDP to collect the data necessary to reach LLM-level generalization in robotics.
2. Why only 224×224 resolution?
They found that increasing image resolution to 384×384 didn't improve performance in their experiments and also took 3x longer to train, so they just stuck with 224×224.
I guess this worked for the robot manipulation tasks within their experiments, but I couldn't help but wonder what happens when a task depends on small/distant visual details? For example, what if a robot were instructed to shoot a Nerf gun at a faraway target?
dw I'm not trying to build a terminator.
3. LoRA and generalization
OpenVLA was the only approach to achieve at least 50% success across all tested downstream tasks in their fine-tuning experiments, making it a strong general-purpose option. However, Diffusion Policy still produced smoother and more precise trajectories for narrower but highly dexterous tasks.
They also found that LoRA nearly matched full fine-tuning while updating only 1.4% of parameters, taking just 10-15 hours on a single A100, an 8x reduction in compute. I realized I should probably read up on LoRA soon after seeing this.
4. A surprising quantization result
Their inference results initially confused me:
bfloat16: 71.3% success, 16.8GB
int8: 58.1% success, 10.2GB
int4: 71.9% success, 7.0GB
Why was int4 matching bf16 while outperforming int8?
It turns out int8 introduced inference overhead that slowed the control loop to 1.2Hz, while int4 reached around 3Hz. The performance difference was largely due to control latency rather than quantization accuracy itself.
This is definitely something I'll experiment with when quantizing models for edge inference.
Overall, reading OpenVLA kinda confirmed how important scaling is in robotics, and made me more optimistic about how combining generalist robot policies with higher-level agentic systems might actually work out.
From reading this paper, I also realized I want to revisit LoRA, FlashAttention, and speculative decoding to better understand large-scale GPU training and inference optimization. I went through some of these ideas while reading through the DeepMind scaling book this past summer, but there's still a lot I want to understand more deeply.
@Cortixofficial You’re right, but that is still bottlenecked by how many people we can record, so it’ll be hard to scale it as much as llms which aren’t even that robust in the first place. I guess potential solutions include rl since that’s bottlenecked by compute
Day 21/365 of robot learning.
I finally finished reading OpenVLA today! Unlike ACT or Diffusion Policy, the paper focused less on developing new algorithms and more on scaling existing architectures, improving data mixtures, and optimizing training, so it was pretty easy to get through.
It almost felt like a better application of the Bitter Lesson to robotics, borrowing a lot of the approaches that made LLM scaling successful.
I also couldn't help reading bits of π0.7 after watching Sergey Levine's recent conversation with Aditya Agarwal about zero-shot generalization and adapting to new tasks through simple language instructions rather than task specific fine-tuning, which OpenVLA relied on for new setups.
Here are a few key takeaways and thoughts on OpenVLA:
1. Scaling robot learning
OpenVLA was pretrained on 970K robot demonstrations, carefully filtering and reweighting datasets based on task diversity and performance. Surprisingly, they trained for 27 epochs, unlike the 1-2 epochs typically used for LLM/VLM pretraining.
It feels like scaling laws are starting to kick in, but we're still bottlenecked by how much human demonstration data we can collect. This made me wonder whether we even have enough global GDP to collect the data necessary to reach LLM-level generalization in robotics.
2. Why only 224×224 resolution?
They found that increasing image resolution to 384×384 didn't improve performance in their experiments and also took 3x longer to train, so they just stuck with 224×224.
I guess this worked for the robot manipulation tasks within their experiments, but I couldn't help but wonder what happens when a task depends on small/distant visual details? For example, what if a robot were instructed to shoot a Nerf gun at a faraway target?
dw I'm not trying to build a terminator.
3. LoRA and generalization
OpenVLA was the only approach to achieve at least 50% success across all tested downstream tasks in their fine-tuning experiments, making it a strong general-purpose option. However, Diffusion Policy still produced smoother and more precise trajectories for narrower but highly dexterous tasks.
They also found that LoRA nearly matched full fine-tuning while updating only 1.4% of parameters, taking just 10-15 hours on a single A100, an 8x reduction in compute. I realized I should probably read up on LoRA soon after seeing this.
4. A surprising quantization result
Their inference results initially confused me:
bfloat16: 71.3% success, 16.8GB
int8: 58.1% success, 10.2GB
int4: 71.9% success, 7.0GB
Why was int4 matching bf16 while outperforming int8?
It turns out int8 introduced inference overhead that slowed the control loop to 1.2Hz, while int4 reached around 3Hz. The performance difference was largely due to control latency rather than quantization accuracy itself.
This is definitely something I'll experiment with when quantizing models for edge inference.
Overall, reading OpenVLA kinda confirmed how important scaling is in robotics, and made me more optimistic about how combining generalist robot policies with higher-level agentic systems might actually work out.
From reading this paper, I also realized I want to revisit LoRA, FlashAttention, and speculative decoding to better understand large-scale GPU training and inference optimization. I went through some of these ideas while reading through the DeepMind scaling book this past summer, but there's still a lot I want to understand more deeply.
Day 20/365 of robot learning.
I spent today experimenting with @dreamlabs_ai Dream-1 model on my so101 arm, trying to get it to complete a simple pick-and-place task without any fine tuning.
Since I'm running a single arm on my macbook instead of the typical two-arm setup the model was designed around, I had to configure everything manually.
After 20+ runs, a few changes somewhat improved its performance:
-Experimented with mapping my so101 as the model's left arm instead of the right, which lead to fewer hallucinations.
-Checked the gripper's tcp with dreamcli tcp; it needs to sit right at the fingertips (moving it inward into the gripper made things worse).
-Repositioned my top and wrist cameras to better match the YAM setup the model was trained on.
-Added table collision constraints and joint limits to prevent the arm from pressing into the table.
In my best run so far, the arm reached my deodorant, but the gripper closed at the tip of it. This was the same issue I encountered in sim funnily enough. It worked there, but not in real life. It still hasn't completed the full pick-and-place task, but it's definitely making some progress!
Also, huge thanks to @NamraByTheSea from Dream Labs for reaching out and helping me debug the setup! @dreamlabs hoping I can get some extra credits since the API wasn't working with my single arm setup :)
Going to get back to reading papers soon and try out @nvidia 's newly open-sourced ASPIRE framework to experiment with agentic robot control!
Day 20/365 of robot learning.
I spent today experimenting with @dreamlabs_ai Dream-1 model on my so101 arm, trying to get it to complete a simple pick-and-place task without any fine tuning.
Since I'm running a single arm on my macbook instead of the typical two-arm setup the model was designed around, I had to configure everything manually.
After 20+ runs, a few changes somewhat improved its performance:
-Experimented with mapping my so101 as the model's left arm instead of the right, which lead to fewer hallucinations.
-Checked the gripper's tcp with dreamcli tcp; it needs to sit right at the fingertips (moving it inward into the gripper made things worse).
-Repositioned my top and wrist cameras to better match the YAM setup the model was trained on.
-Added table collision constraints and joint limits to prevent the arm from pressing into the table.
In my best run so far, the arm reached my deodorant, but the gripper closed at the tip of it. This was the same issue I encountered in sim funnily enough. It worked there, but not in real life. It still hasn't completed the full pick-and-place task, but it's definitely making some progress!
Also, huge thanks to @NamraByTheSea from Dream Labs for reaching out and helping me debug the setup! @dreamlabs hoping I can get some extra credits since the API wasn't working with my single arm setup :)
Going to get back to reading papers soon and try out @nvidia 's newly open-sourced ASPIRE framework to experiment with agentic robot control!
@karthikvetrive3@dreamlabs_ai I followed the regular setup guide to run it in mujoco. But you can also check out the sandbox, they already have everything configured for you
Wasn't industry supposed to scale the main line of theory while academia explored the branches?
But robotics is taking way too many branches even in industry 😭
Is this what happens when academia leads industry or just a sign of rapid innovation?
ig the architecture wars aren't ending anytime soon
@adityaag Agreed, this is helpful to hear. I've been debating whether to pursue a phd after undergrad or go all in on applying and scaling current robot learning approaches through industry research
We won 2nd place @ycombinator's Make Something Agents Want Hackathon in the @insforge track!
We built Cortex: Cursor for training robot arms autonomously. After facing the tedious process of training a policy from scratch at @HackMIT last month, we instantly got to work on finding a faster method. Cortex lets agents control the whole pipeline, from data collection to deployment.
Big thanks to my teamates @praneethsamine and @suhasdev23 for building with me. Also, thank you to @nemild, @hanghuang_, and @tonychang430 for organizing the hackathon and track.