SkyTrack

An open platform to build and scale real-world autonomous mission across multiple vehicle types

[Marco Franzon @mfranz_on]

The future of agriculture lies in optimization. A system like this costs very little: a drone with a camera and a computer vision system accurate enough to count and identify plants or diseases.

It's so easy that anyone can make it at home for their own crops.

[SkyTrack @GetSkyTrack]

One drone planting one tree is a cool demo. A fleet doing it across thousands of hectares?
Interesting to dive into how mission logic enables consistent outcomes whenever a fleet gets sent to the field.
#missionapp #drone

[Li Zexin 李泽欣 @XH_Lee23]

China uses drones to plant trees and green the barren mountains.

[SkyTrack @GetSkyTrack]

Spot on. Same dynamic on the aerial side, except the unpredictable variable is the atmosphere, not the contact surface.
Sim runs flat.
Reality runs varied.
Field is where the pre-built sim gets stress-tested by wind, GPS drift, and thermals.
#simtoreal #UAV

[Eren Chen @ErenChenAI]

Why Robots work in Simulation but fail in Reality

One of the most frustrating moments in robotics:

Everything works perfectly in simulation. Then you deploy it on a real robot and suddenly:

The grasp misses
The arm shakes
The robot drifts
Contact becomes unstable
The motion looks correct, but the task still fails

How do you solve the sim to real problem?

At first, it sounds simple. Just move the code from simulation onto hardware.

But the gap between simulation and reality is much larger than most people think.

Simulation environments are extremely clean.

The table is flat.
Object geometry is accurate.
Friction is predefined.
Sensors are stable.
Robot joints behave exactly as expected.

But the real world is messy.

Lighting changes.
Depth sensors drift.
Objects reflect light differently.
Motors have delay.
Joints have backlash.
Contact forces behave unpredictably.
And robotics is a chain reaction.
A small perception error becomes a planning error.
The planning error becomes a control error.
The control error becomes an execution error.

Eventually, the robot misses the grasp by a few centimeters and the entire task fails.

And The hardest part is usually contact.

Humans think tasks like: grasping a cup, opening a door, inserting an object, pushing a box are trivial.

For robots, these are extremely difficult because contact is not clean physics. A tiny shift in friction, force, or surface geometry can completely change the result.

In simulation, objects are usually “well behaved.”
In reality:
objects slip
contact points shift
surfaces deform
collisions happen unexpectedly

This is why many robotic tasks fail not because the policy is fundamentally wrong, but because reality itself introduces uncertainty.

Sensors are also less reliable than people think.

The robot’s perception already contains error:
camera noise
unstable depth estimation
occlusion
pose estimation drift
changing lighting conditions

Sometimes the model itself is fine, but the input is already slightly wrong. By the time the error propagates to the end effector, the grasp fails.

The robot hardware itself is also imperfect.
Motors have latency.
Controllers have frequency limits.
Actuators have error.
Different loads change behavior.

In simulation, the robot follows commands perfectly.

In reality, it may move slightly slower, slightly off target, or slightly unstable. Those tiny differences are fatal in robotics because robots physically interact with the world.

Sim2Real being difficult does not mean simulation is useless. Simulation is still incredibly valuable: they are cheap, safe, scalable and reproducible.

A better way to think about simulation is: Simulation is the training ground, not the final battlefield.

Modern Sim2Real methods usually combine multiple approaches: making simulation more realistic, adding domain randomization, randomizing lighting, friction, object positions, and sensor noise, fine-tuning with real-world data.

The goal is not to make the robot adapt to one perfect virtual world. The goal is to make the robot robust enough to survive an imperfect real one.

The most important lesson in robotics is:

Success in simulation is only the first step. The real test begins when the robot touches the real world.

Video Credit: Kevin Zakka

[SkyTrack @GetSkyTrack]

→ Mission logic tested independently of hardware
→ Missions reused across environments
→ System behavior reasoned about before deployment

The same logic travels across agriculture, logistics, inspection. Read more https://t.co/QLj5dP0ptV

[SkyTrack @GetSkyTrack]

A mission application, defined properly, has three parts:
→ A clear objective
→ Explicit constraints — safety, timing, coverage
→ Observable states and transitions over time
Shifting from hardware-led to mission-led makes three things possible 👇

[SkyTrack @GetSkyTrack]

Early access is widely open.
Some of it's rough, some of it's fast, and we're humbly looking forward to your first sim building.
Try it out → https://t.co/IxpJCx5xyo

[SkyTrack @GetSkyTrack]

Imagine writing your UAV mission logic once — and deploying it across different drones, fleets, and operational contexts.
That's what we've been building. We're calling it SkyTrack.
Take a look 👇 https://t.co/nC0VXoOpd8

[SkyTrack @GetSkyTrack]

What if the drone was never the point?
For a decade, UAV innovation has been hardware-led. Better airframes. Better sensors. Better autopilots. All of it useful.
The next frontier is mission logic that makes the hardware independence possible.