mortenlysgaard

Lead vision engineer at @1x_tech, former co-founder CTO at interested in learning based robotics, embedded systems, vision etc.

[Seohong Park @seohong_park]

Behavioral cloning mystery

https://t.co/VqxzvcfGSx

I wrote a new blog post about "mysteries" in behavioral cloning that appear with real-world robot data (e.g., overfitting is "good"). I also tried to demystify them and shared my thoughts!

[mortenlysgaard @mortenlysgaard]

@VilleKuosmanen Working on gripper designs at the moment, would love to join

[mortenlysgaard @mortenlysgaard]

@sirwart How about GIGe?

[mortenlysgaard @mortenlysgaard]

@enactic_ai @pravsels Will you sell the modules themselves?

[mortenlysgaard @mortenlysgaard]

@LeRobotHF What hardware (gpu) is used for pretrain and finetune?

[mortenlysgaard @mortenlysgaard]

@chris_j_paxton Check out High Torque robotics Panthera arm. I have it. it’s fast, strong, relatively torque transparent, I’ve not been able to break it yet and it’s cheap!

[High Torque Robotics @HTrobotics]

🚀 Panthera-HT is here — the cost-efficient robotic arm that doesn't compromise performance. Only $1,799.3.5kg payload | 0.8m/s speed | ≤140μs response Fully open-source (MIT) | LeRobot-native | ROS2 ready

#Robotics #OpenSource #EmbodiedAI #LeRobot #XLeRobot

[mortenlysgaard @mortenlysgaard]

@VectorWang2 @chris_j_paxton Is this design open? I alread have a pair of High Torque Panthera arms, and they’re great. Would be cool to make them mobile!

[mortenlysgaard @mortenlysgaard]

@KuphDev @theotime_lh That test case look like it could be solved just playing out a trajectory open loop?
The model can only demonstrate that it beats trajectory playback if there is some sort of variation in the problem, e.g. random initial duck placement and orientation.

[mortenlysgaard @mortenlysgaard]

@quantbagel Using query tokens instead of spatial pooling is a great insight! Are you planning to release more details?

[mortenlysgaard @mortenlysgaard]

@Vikashplus @ChongZitaZhang When humans thread a needle, they reduce the DOF of their arms+hands greatly by fixing and leaning onto things. We use pinch grips so that opposing finger force implicitly controls position with high precision. I think force control is much more important than position.

[Bernt Bornich @BerntBornich]

These guys get it (equally true for rest of robot, for safety and ability to learn thru failure, not just sim2real gap)

Adding NEOs hands:

DOF: 22 (44 active tendons per hand, fully actuated)
Ratio: 8:1 (w/tendons, 1X custom high-torque motors)
Sim2Real Gap: Low (10-15% friction, high stiffness)
Force Transparency: High (motor currents)
Reliability: High (3.5m cycles at nominal load)

[mortenlysgaard @mortenlysgaard]

This!

[Mehdi 麦迪⁠ @BetterCallMedhi]

I spent time in Shenzhen last year and when I saw Merz come back from China saying Germans need to work more I immediately knew what broke his brain because I lived the exact same cognitive shock

my first week in Huaqiangbei I burned through 4 prototype iterations of a motor controller board for less than a thousand bucks total, back home a friend was working on something similar and spent over 12 thousand for a single revision that took almost two months to arrive

when you live that contrast in your own hands with your own project something permanently shifts in how you see the world and it goes way deeper than speed & cost

what Shenzhen actually built is a collective learning organism, imagine 20 PCB fabs 15 injection mold shops 30 component distributors and a hundred firmware freelancers all within a 2km radius, looks insanely redundant from the outside until you realize redundancy is actually information density in disguise

I watched this firsthand with an injection mold supplier I was working with, this guy had seen a hundred founders iterate similar thermal designs over 6 months so he proactively modified his tooling before I even opened my mouth, he knew what I needed before I knew what I needed, the intelligence lives in the relationships between the nodes and it compounds daily

the west thinks about manufacturing as a cost center you optimize by centralizing…

China accidentally built a distributed neural network of manufacturing intelligence where knowledge diffuses horizontally across thousands of agents faster than any single western company can process internally

so when Merz comes back and says we need to work a bit more I think he saw the problem but COMPLETELY misdiagnosed the solution, telling Germans to work harder is like telling a horse to gallop faster when the other side built a combustion engine

the gap is ARCHITECTURAL

it’s ecosystem density, you need a custom connector in Shenzhen you walk 200 meters, in Munich you send an email and wait 3 weeks

it’s iteration speed, parallel search vs sequential optimization at the system level, it’s risk tolerance, Chinese founders ship something broken on Monday fix it Tuesday ship again Wednesday while European companies are still in the approval phase for the pilot program of the feasibility study…

and Merz only saw the surface, what he missed is the tier 2 cities like Hefei Chengdu Wuhan replicating the Shenzhen model at scale right now

BYD going from irrelevant to outselling every european automaker combined in roughly 5 years, Huawei building its own 7nm chip under maximum sanctions when every analyst said it was physically impossible & behind all of that a government that treats advanced manufacturing as an existential national priority while europe debates whether AI needs another ethics committee

I think what we’re watching is the most asymmetric economic competition in modern history and most western leaders are still framing it as a productivity problem when it’s actually an ontological one

Europe & America are optimizing variables that China stopped tracking years ago meanwhile China is compounding on dimensions the west has no framework to even measure
Merz at least had the courage to name

it out loud and I respect that genuinely but working a bit more inside a broken architecture just means you arrive at the wrong destination slightly faster

[mortenlysgaard @mortenlysgaard]

@DominiqueCAPaul Missing IR filter?
Mixup of yuv and rgb some point in the pipeline?
It seems to extreme to be a pure calibration issue, but that is also possible. Full calibration is quite an involved process though.

[mortenlysgaard @mortenlysgaard]

@DominiqueCAPaul There is a mixup between BGR and RGB color ordering somewhere. OpenCV for some reason insists on BGR for many of its functions, which often is surprising and gives this result.

[mortenlysgaard @mortenlysgaard]

@VectorWang2 @chris_j_paxton Have you tried to lift heavier things? Eg 1kg, does the dynamics change a lot when it is loaded or is it still tight?
Is there much backlash?

[mortenlysgaard @mortenlysgaard]

@VectorWang2 @chris_j_paxton How is your experience with the panthera from high torque robotics?

[mortenlysgaard @mortenlysgaard]

@chris_j_paxton TV and social media are not bad in themselves, but, kids, as any living being, have a finite time on their hands. Human development strongly favors building motor skills in age 0-16. This takes a lot of time, and if you do not do it as a kid, you loose out on a lot.

[Ursula von der Leyen @vonderleyen]

EU-Inc.

Making business truly borderless.

[Ning Ding @stingning]

Building upon SimpleVLA-RL, we have implemented real-world RL on long-horizon dexterous tasks and witnessed a non-trivial (~relatively 300%) performance improvement over the SFT model, along with surprising capabilities on auto-recovery. Blog coming soon. The entire process uses very little data and training compute—basically costing no more than a single robotic arm—hinting that real-world generality for machines is actually within sight.

[mortenlysgaard @mortenlysgaard]

@builds_robots Awesome demo! <3 I see you've both used i2rt YAM and agilex piper, any experiences to share about how they are to work with?