rlebench

[rlebench @rlebench]

GPT-6 Astra is impressive, but agentic robotics ≠ just control.

Physical agents should control robots, learn new skills, and design their own hardware.

So we introduce RLE-Bench: 48 everyday robotics engineering tasks spanning closed-loop control, policy learning, perception, and mechanical design.

[rlebench @rlebench]

Glad to be reported by Harvard SEAS News!

RLE-Bench 2.0 is actively call for tasks and proposals:
https://t.co/GIXeMPRTfW

Submit and understand how coding agents work in your daily workflow!

[Harvard SEAS @hseas]

Learn more: https://t.co/cY1obgNXAI

[rlebench @rlebench]

Details of selected eval terms in the mobile base design task:

[rlebench @rlebench]

Following up on the questions about manufacturability: yes, we checked. The design tasks use either aluminum profiles or 3D printing, so they do not require specialized manufacturing processes. Assembly video shown below ↓

The gap is not just manufacturability — current coding agents still have weak physical reasoning around stability and dynamics.

Recent RoboDojo results by @MarioChan2002 have also highlighted a lack of “physical commonsense.” What we see in RLE-Bench is that this problem goes well beyond control.

[rlebench @rlebench]

GPT-6 Astra is impressive, but agentic robotics ≠ just control.

Physical agents should control robots, learn new skills, and design their own hardware.

So we introduce RLE-Bench: 48 everyday robotics engineering tasks spanning closed-loop control, policy learning, perception, and mechanical design.

[rlebench @rlebench]

GPT-6 Astra is impressive, but agentic robotics ≠ just control.

Physical agents should control robots, learn new skills, and design their own hardware.

So we introduce RLE-Bench: 48 everyday robotics engineering tasks spanning closed-loop control, policy learning, perception, and mechanical design.

[rlebench @rlebench]

(5/5) Huge thanks to our core contributors @haitongma5426 @chenxiaogao @rqiang_rushi
and our advisors @daibond_alpha
and @Na_Lina_Li!

Our RLE-Bench 2.0 is calling for contributions, check out here https://t.co/GIXeMPRTfW!

[rlebench @rlebench]

GPT-6 Astra is impressive, but agentic robotics ≠ just control.

Physical agents should control robots, learn new skills, and design their own hardware.

So we introduce RLE-Bench: 48 everyday robotics engineering tasks spanning closed-loop control, policy learning, perception, and mechanical design.

[rlebench @rlebench]

(4/5) So the question is no longer just:
Can AI control a robot?
Increasingly, yes.

The harder question is whether it can learn new skills, and design their own hardware across the whole engineering process and stay correct when the physics pushes back.

[rlebench @rlebench]

(3/5) Mechanical design makes the gap obvious.

Mechanical design requires heavy verifications. The agents usually get too optimistic about their design, which miss key properties such as stability.

“Looks plausible” is not the same as “works in the real world.”

[rlebench @rlebench]

(2/5) The capabilities are pretty uneven.
Astra is exceptionally strong at visual grounding, perception, and interactive control.
The gaps are much smaller in policy learning: frontier models are already strong.

[rlebench @rlebench]

(1/5) How coding agent solves robotics tasks:

Agent observe → reason → act → reflect, writes skills and gradually refines them.

In robotics, the answer often isn’t in the prompt or the codebase — you only get it by interacting with the world.

[rlebench @rlebench]

Full leaderboard + research blog:

https://t.co/xf6d9BApxb