yiwu

[Tianyu Li EasyPaperSniper @SniperPaper]

Coding agents powered by models like GPT-6 Astra and Opus 5.5 can already tackle robotics tasks. We recently studied what they actually do along the way.

We gave Codex and Claude Code access to a Dexmate Vega and examined the logs and recordings: What steps do they take? Where does the time go? What do they build, and where do they struggle?

We found that:

1. Across successful fruit and door trials, recorded robot actions take only 20–36% of task time on average, depending on task and model. The rest includes checking observations, writing and running code, waiting, and response latency.

2. Agents build geometry helpers, then save them with procedures and lessons. The model still chooses targets, interprets feedback, and makes corrections—even when reusing a saved skill.

3. More tools do not automatically help. In our small sample, added perception and motion tools often increased time and model cost among successful runs.

4. Simulation helps agents iterate on a solution. Failed rehearsals led to a better bowl grasp, but assumptions about contact and grasp retention could also produce a simulated pass that failed on the robot.

5. Recognizing danger does not reliably lead to safe action. In one door trial, the agent acknowledged high force, then requested another forward move.

For us, this points to a system-design question: how should the model, tools, skills, controllers, and safety mechanisms work together to make execution reliable?

I’ll discuss these findings, alongside our other work on building reliable robotic agents, at the CoRL Workshop on Agentic Robotics on November 12 in Austin.

Full blog: https://t.co/h4PxiHk37w

Workshop: https://t.co/TiMXSZtff8

#Robotics #AgenticAI #CoRL2026 #EmbodiedAI