Pau Gargallo

Reconstructing the world one image at a time at prev @RealityLabs and @mapillary

[Pau Gargallo @paugargallo]

@jon_barron Hans Rosling vibes on this one

[Pau Gargallo @paugargallo]

@dhh Awesome! Thanks! Did you manage to turn off the led strip behind the screen? I failed at it and ended up putting black tape on it 😅

[Pau Gargallo @paugargallo]

@DominiqueCAPaul @DominiqueCAPaul, i love following your adventures. Thanks for sharing them openly. We are building https://t.co/t9o6XLnezr to drive robot fleets. Would love to meet with Zurich based founders.

[Pau Gargallo @paugargallo]

@DominiqueCAPaul 🙋‍♂️

[Pau Gargallo @paugargallo]

@DanielMiessler So cool! What is the voice to text tool you are using with CTRL+J at 6:30 in the video?

[Nikhil Keetha @Nik__V__]

Meet MapAnything – a transformer that directly regresses factored metric 3D scene geometry (from images, calibration, poses, or depth) in an end-to-end way. No pipelines, no extra stages. Just 3D geometry & cameras, straight from any type of input, delivering new state-of-the-art results 🚀

One universal model enables SoTA for:
🔥 Mono Depth Estimation
🔥 Multi-View SfM
🔥 Multi-View Stereo
🔥 Depth Completion
🔥 Registration

… and many more possibilities! – plus everything is metric 🎯

We release code for data processing, training, benchmarking & ablations – everything Apache 2.0!

Details & Links 👇

[Pau Gargallo @paugargallo]

@AjdDavison Anything with inputs too large to fit into memory?

[Pau Gargallo @paugargallo]

@iquilezles @DoItRealTime Can you get more phases by rotating the gears wrt each other?

[Pau Gargallo @paugargallo]

You know you are an old CV fart when you see SFM and think "Structure from Motion", but they mean "Spatial Foundation Models"

Cool stuff: https://t.co/ZWiwYqF08S

[Fang Roffo 🎗 @fangr]

@abierkhatib

[Shreyas Gite @shreyasgite]

Gemini + π0 = actually useful robots! (Similar to what @physical_int did with "Hi Robot")

I can now verbally tell the robot that I'm building a red Lego wall or wooden tower, and it will infer the next steps by itself and pass me the necessary pieces, tools, or materials, ha!
You can also just ask it to bring you things!

The pipeline works as follows:
- OpenAI Whisper (local) → speech to text
- Gemini → makes sense of user requests, converts to robot tasks, bounding boxes, grasping points, etc. (System 2 thinking FTW!)
- π0 → robotic actions

The π0 was finetuned just for pick-and-place Lego bricks only, and it generalizes beautifully to all kinds of tasks. However, there's lots of room for improvement when it comes to grasping & accuracy.

Things that could help:
- Conditioning on grasping points
- Better data collection (I'm not that great at teleop)
- Lots more synthetic data and simulations

[Boyang Deng @boyang_deng]

Thought about generating realistic 3D urban neighbourhoods from maps, dawn to dusk, rain or shine? Putting heavy snow on the streets of Barcelona? Or making Paris look like NYC? We built a Streetscapes system that does all these. See https://t.co/pOafp2fWlv. (Showreel w/ 🔊 ↓)

[Pau Gargallo @paugargallo]

@sellan_s @YouJiacheng Cool thanks! I couldn’t avoid seeing a sphere that was inside the curve in one drawing and outside in another and had to ask 😅

[Pau Gargallo @paugargallo]

@sellan_s @YouJiacheng do you also use the normals during the reconstruction step to ensure that the surface is tangent to the spheres?

[NAVER LABS Europe @naverlabseurope]

The wait is over 📢 MAST3R is out! DUSt3R+ dense local feature maps & metric depth - 1st in #MapFreeReloc leaderboard, can handle 1000s of images 😀 !!
Blog: https://t.co/gX7ez0fk93
Code: https://t.co/qPcOYTD3Hk
Paper: https://t.co/iq750lQIJK

[Pau Gargallo @paugargallo]

@jon_barron Rage Against the Machine

[UploadVR @UploadVR]

The Quest v64 update brought two undocumented major new features: furniture recognition on Quest 3 and simultaneous hands & controllers in the home space:

https://t.co/Kw4ZAhqwLB

[Pau Gargallo @paugargallo]

@karpathy I’ve only heard about Karpathy compile doing this

[Mapillary @mapillary]

Now available on Mapillary: Neural Radiance Fields (NeRFs)! 🎊

NeRFs allow for the transformation of a collection of 2D images into detailed, immersive 3D reconstructions.

Read our blog post to learn more and see how you can get started with NeRFs: https://t.co/UwcvHpuFYz

[Pau Gargallo @paugargallo]

@toni_mateos Felicitats, cracks!