flexigrip uses soft robotics to handle fragile objects
without damage pastries, irregular packages, delicate components, same gripper
rigid automation fails at variety compliant mechanisms dont
There's been a few cool updates recently. In particular, @rerundotio 0.33 released headless rendering. This, along with the Fable 5 release pushed me to work torwards making MAMMA realtime!
I threw Fable at the problem, and it was able to take original implementation that was ~12 seconds / frame and get it all the way down to 40ms /frame, or nearly a 300x speedup 🏎️
How did I achieve this?
TLDR:
- Use rerun's headless rendering as supervision when optimizing
- Save rrd file as test fixture to guide model optiziation with /goal
- create an html artifact with headless rendering to provide detailed breakdown of what it did and how it actually looks like in the viewer
There were a few critical bits to make sure that this ACTUALLY worked and that Fable didn't just cheat or delete something and declare victory.
The first is that the original version used Rerun, this allowed us to save things to disk as an RRD file, meaning we could query the contents and use this as a sort of test fixture or golden artifact that held EXACTLY what all of the values should be. Then we can use this with /goal as a metric when doing the optimization to ensure there are no regressions.
The second bit is the headless rendering, this gave us the ability to check that not only did the test fixture pass, but it also looked visually correct. This made a huge difference, and an awesome side affect of it is that we can use the headless rendering to create an implementations.html file. This gives a visual guide as to what the agent did (I walk through it in the video below)
Along with this, we're working on an MCP server for rerun that allows full interactivity with the rerun viewer for your agent. So for example the agent can click, drag, move views, scroll timelines, ect. I used this to help the agent debug certain parts such as when the 2d sam masks didn't line up, or if the triangulated keypoints werent correctly matching with the optimized mesh. The agents could go, click into the view, scroll through the timeline and see where things went wrong.
Fable + Headless Rendering + Rerun MCP == 300x speedup in less then a days work
With these new tools, I'm planning on going back to my gaussian splatting implemntation and cleaning it up + making it fast!
.@ValigurskyM shared a 3DGS reconstruction of downtown Lublin, Poland, built from 250 million splats and viewable directly in your browser.
Check it out: https://t.co/BRmXKH8fIi
We present Wild3R for Feed-Forward 3DGS in the Wild🦁
Now, we can reconstruct appearance-consistent 3D scenes from unconstrained photos in a second🔥
Project page: https://t.co/SoG2UWALqj
ArXiv: https://t.co/lA3SWPezjp
Mounted Mighty Camera on a $20 toy drone for a short circular flight in backyard. Just to see how well it works against motor vibrations (I did try some minimal dampening).
It works well until some vibrations slip in and make IMU noisy 😄
🏀 Arcade Hoops is now available on the official Meta Horizon Store! 🚀
This was such a fun test: taking an idea and shipping it to the Meta Horizon Store in exactly 4 weeks.
I built it using:
- Meta VR CLI
- Agentic Tools Repo, full of skills
- Claude Code + VS Code
- Unity 6 + Unity MCP
- Meta XR Core + Meta ISDK + MRUK
- OpenAI Image 2.0 for concept generation
- Adobe Photoshop for all the store assets
💡Honestly, this project taught me so much about what modern VR/MR development workflows can look like when you combine Unity with our agentic workflows.
A fun way to try the new Claude Fable. I asked it to train a bot using RL to navigate 3D worlds created with @SpAItial_AI.
This is all running in the browser including RL training! Link to demo and code in comments
What you can do:
-> Train a bot from scratch, watch it learn in minutes, then drop it into a world it has never seen and it keeps navigating obstacles.
-> The worlds are real scenes generated from a text prompt or a photo via the @SpAItial_AI API (you get a splat + a collision mesh).
-> Mesh is made into a 2D walkability + height grid that the RL environment runs on.
-> The trick to generalization: the policy only ever sees egocentric inputs. A 16-ray lidar, the goal's body frame, and its speed. No global coordinates, no map.
So it learns navigation, not a memorized world.
With flashdreams, you can also create your own game, without paying anyone! Entirely open-sourced with Apache 2 license.
All from a text prompt and a single image
https://t.co/VOy1ir4o1D
Playing with omnidreams (https://t.co/OgezYwTETB) inside flashdreams (https://t.co/ALI6NBJYnn)
Fully interactive video generation at
- 30FPS,
- 720P resolution,
- On a single RTX 6000 PRO GPU
Gaussian splats are captures frozen in time — not anymore. ⚡
Experimental RELIGHTING of splat scenes in the @playcanvas engine, driven by a proxy mesh: swap the sky, drag the sun with its soft shadows, drop in point lights — all live on a captured scene.
🔗 Runnable demo👇