This is so true for anything involving robotics and automation… and it’s a big part of what the @USAvionix team is thinking about.
I’m particularly interested in what happens when drones evolve from autonomous vehicles executing predefined missions into truly agentic machines.
Today, humans still sit in the middle of many of the decisions these machines make. And that creates a fundamental bottleneck.
A drone can generate and receive enormous amounts of information continuously… imagery, telemetry, weather, changing conditions on the ground, information from other aircraft and robots, and changing mission objectives.
But humans can only process a fraction of that information. And our throughput for turning information into decisions and actions is inherently limited.
So incredibly fast machines end up operating at human decision-making speed.
The future is different. Machines will increasingly perceive, reason, coordinate and adapt in near real time… with other machines in the air, robots on the ground, and continuous streams of information. Humans move higher in the loop, defining intent, objectives and guardrails rather than coordinating every individual action.
But that creates another problem: how do you safely train and test machines that are making thousands of decisions across constantly changing environments?
This is where what Lukas is describing gets really interesting.
High-fidelity 3D representations of the physical world give us a way to bridge bits and atoms. Autonomous systems can experience and be tested against millions of changing conditions, interactions and edge cases in simulation before encountering them in the physical world… and what happens in the physical world can continuously feed back into better simulation, testing and autonomy.
Physical world → simulation → testing/training → physical deployment → new real-world data → better simulation → better autonomy.
And something else has changed dramatically: the cost and speed of creating these environments.
There are obviously many sophisticated simulation tools already out there. But I think the combination of really good prompts, tight development loops, and teams of AI sub-agents working with accessible tools like Three.js is going to let us build and iterate on increasingly sophisticated 3D simulations far faster and far more economically than we’ve been able to before.
It’s also one of the reasons I’ve personally gotten so deep into building flight simulations inside 3D environments using @threejs. What started as experimentation has increasingly made me realize how powerful these environments can become for testing autonomy, coordination and decision-making before bringing those behaviors into the physical world.
We’re still early… but I think the line between simulation and the physical world is going to get very blurry…
Personal update: I've joined Anthropic. I think the next few years at the frontier of LLMs will be especially formative. I am very excited to join the team here and get back to R&D. I remain deeply passionate about education and plan to resume my work on it in time.
lmao I can't stop laughing
claude-code has a "Frustrated User Detection"
There's a regex that detects when you're angry
( fully hard coded btw)
When triggered, it changes Claude's behavior/UI state.
Claude literally knows when you're cussing at it.