Autelica

A virtual camera for real moments. Three small drones film one moment from three sides; you choose the camera afterwards. Multi-angle edits, stats, 4D replays.

[Autelica @autelica]

Introducing Autelica!

We're building a virtual camera and 4D replay for live-action sequences: three aerial cameras film one moment from three sides, and you choose your own trajectory through the action afterwards.

This is our interactive lab, running on real footage from three tripod cameras. Drone flights start this week.

[Autelica @autelica]

@nikunj Too real!

[Autelica @autelica]

4/ Thanks to @leggedrobotics (ETH RSL) for open-sourcing both the dataset and the policy.

Read more here: https://t.co/tkymzNsVhV

[Autelica @autelica]

How close do ordinary cameras get to lidar under a walking robot's feet?

We checked against ~31,000 real footfalls from ETH's GrandTour dataset. On natural ground, 90% land within 8.6 cm of our camera terrain. Lidar: 6.3 cm. Guessed flat ground: 34 cm.

[Autelica @autelica]

3/ Where lidar wins: sunny ground with the robot's own shadow, wet concrete, metal grating, dark floors. In tall grass neither sensor finds the ground the foot lands on. All on the page, with the protocol.

[Autelica @autelica]

2/ Does a few cm matter? We ran ETH's open Watch Your Step policy for the G1 on a table we measured. Stepping down: 91% success on measured terrain, 66% when its map is guessed from motion. A sim built on the guessed ground says 100%.

[Autelica @autelica]

@daleverett haha @themathgeek13

[Rohan Rao @themathgeek13]

A fun project I've been working on that combines aerial robotics, SLAM/SfM, and 4D reconstruction! Solving and optimizing for complex sequences in dynamic, open world environments and retargeting for humanoids like the @UnitreeRobotics G1 🚴🏃

[Autelica @autelica]

@elonmusk That's why we are building live 4D replays for live sports with spatial AI!

[Ken Goldberg @Ken_Goldberg]

Agentic Robotics -- using AI coding agents to design and fine-tune robot control systems using harnesses like ENPIRE, ASPIRE, or GaP -- are super exciting. If you know of any small or medium manufacturing companies who are applying Agentic Robotics to improve real production, please get in touch.

[Autelica @autelica]

I tracked a run with three cheap drones and trained a simulated humanoid (Unitree G1) to copy it.

The hard part is knowing exactly where each drone was. Using the drones' own GPS, the motion comes out about 72 cm off. Solving all the camera positions over the whole flight brings it to about 6 cm.

The comparison is a simulation with real mocap as ground truth and drone noise taken from my flight. Real-world numbers are next.

[Autelica @autelica]

We're here for it! Building interactive spatial replays of live action sequences! #sportstech #multidrone #aerialcinematography

[Autism Capital 🧩 @AutismCapital]

VR is entering a golden age. Calling it now. AI unlocks the potential for these headsets and immersion now that games can be quickly decompiled and can run on any hardware. If Apple doesn't lock in and deliver a spectacular Apple Vision Pro 2 it would be a biblical level tragedy.

[Autelica @autelica]

Our 3D replays from handheld phone video failed 3 in 10 times, and the same clip gave different answers each run. The culprit: a camera solver treating real-but-weak motion as outliers. One setting later: 10/10, replay in ~3 min. The write-up: https://t.co/T0mkNjazjK

[Autelica @autelica]

@nicochristie @BrainsAndTennis Amazing work! We are doing similar motion tracking and pose estimation through multiple autonomous aerial cameras (drones). If you upload a similar video to our website at https://t.co/yZb1cHLMzd, you can get a similar result and 4D replay!

[Chris Paxton @chris_j_paxton]

What do robot sports teach us?

We have seen incredibly cool progress in robot soccer and tennis recently, but just as interesting is what we definitely do not see.

Robots are increasingly capable of incredibly fluid, reactive whole body control - but their ability to perceive the world while doing so is incredibly limited. And so we see shortcuts being taken, external trackers used, to avoid the limitations of onboard sensing and sensorimotor control while in challenging environments.

The future of robotics will require closing this gap.

Some thoughts ->
https://t.co/7xT0mKyg1I

[Autelica @autelica]

@chris_j_paxton Excited to build in this direction! We have been building markerless capture systems with multiple aerial cameras (drones) that enable 4D replay experiences and also map really well to humanoid robots (we remapped Unitree G1 in simulation for now, aiming for real robots soon)!

[Autelica @autelica]

@azlenelza Love this! Been applying something similar for 4D replays and virtual flythroughs of live action sports scenes - we capture multiple viewpoints with aerial cameras and reconstruct the scene in 3D over time!

[Autelica @autelica]

@virtual_amir @MLS @Apple Super cool stuff! Opus 5.5 is amazing! We have been doing something similar with multi view aerial cameras for live action sports! Capture the moment and replay it in 4D 😎

[Autelica @autelica]

@sohmehta Our product is 4D replay for live action sports! Can you help us out too?

[Autelica @autelica]

@levelsio Haha you're not alone 😂