6/ Real-world task 1: WoollyBallPnP
Base: 14% → After 134 episodes (~15 min): 64%
This task is a relatively simple pick-and-place task, but the base policy fails surprisingly often due to the precision needed not to have the ball slip out of the grasp.
A great summary as usual. I think a lot of people didn't initially get that it was teleop, but I'm personally in the camp that anything which gets robots out there is good for progress; the price is good, even if its teleop.
🎯 A major milestone on the road to embodied intelligence.
This week, @GoogleDeepMind's Gemini Robotics VLA reaches an important milestone in our joint mission to bring AGI into the physical world. By introducing agentic capabilities - the ability to reason, plan, actively use tools, and generalize - we’re moving beyond reactive models and into a new era of robotic autonomy.
We’re proud to be collaborating on Gemini Robotics 1.5 with Google as we gear up to deploy Gemini-powered Apollo humanoid robots in additional customer facilities. It’s a critical step in transforming a powerful model into a field-ready system that performs with consistency, reliability, and purpose.
Read the blog: https://t.co/5Qx8ulADVN
#apptronik #humanoid #humanoidrobot #robotics #google #deepmind #geminirobotics
Figure just highlighted fingertip design as one of their major improvements, and it’s a massively underrated & overlooked part for robotic hands. Im happy they (and others) are now calling it out.
Just because something “looks like” a hand/finger/finger tip doesnt mean it will perform like one
You’ll see demos of hands picking up larger objects or being directly handed tools to manipulate in-hand, but rarely see end-to-end picking something tiny/thin off of a table and using it
That capability comes from the fingertip, and fingertip design isnt easy.
The human finger is so god damn impressive, and it’s impossible to replicate 100% (nonlinear deformities, durometer differences, skin texture/frictional changes, touch sensor capabilities/restrictions) so you end up needing to adjust the design to match capabilities. If you put a glove on yourself (literally any glove) and try to do tasks with smaller/thinner objects, youll notice a significant performance change compared to your own hand and you need to adjust your approach accordingly.
Maybe thats why the figure demo had it picking up stacked plates a little differently than how we would do it as humans (or it could have just been an easier approach for a teleop user/sim to learn)
You have 3 levers to pull for finger tips: shape, compliance, and surface finish
And the touch sensors you choose automatically impose design
constraints on some of these levers.
Depending on your benchmarks, youll start to shift those levers - which can start to lead you away from exact human fingertip shape
It’s interesting looking at the fingertips of hands that come out to see what changes they made to improve capabilities or if their only benchmark was “make look human”. You can really tell just from the finger tips what the robot is going to be useful for.
“Moving on to more general aspects of humans and what we sense as we manipulate, on top of that skeletal muscles sense forces that they are applying or that are applied to them. Muscle spindles detect muscle length and when they stretch, and Golgi tendon organs sense tension in the muscle and hence sense force being applied to the muscle.
We also make visual and touch estimates about objects that change our posture and the forces we apply when manipulating an object. Roland Johansson (again) describes how we estimate the materials in objects, and knowing their density predict the forces we will need to use. Sometimes we are mistaken but we quickly adapt.”
These are the core pillars of our Multi-Force Ergonomic Haptics philosophy underpinning the Contact CI Maestro
“Dexterity is all about moving objects intricately through contact. The sense of touch is critical for this. Vision can help you acquire objects, but anything more complex will need touch.”
Showcasing Maestro ‘twas a great time @ last week’s near frontier hardware fair
sunset backdrop, the overall energy and passion in the gundo certainly didn’t disappoint. S/o @jcbthnflrs for the epic photos while being hands-on w/ our haptics in VR
Thanks for inviting @Contact_CI 🙌
@sofi_a@IanRountree@cantos