Seldon

[Seldon @seldon_tech]

How good are agents actually at CAD?

Today we release CADBench, evaluating 10 Models on 105 realistic tasks in Fusion 360

Fable 5 and Gemini 3.7 Flash are #1 with only 24.6% pass rate

[Pascal Bertrand @PascalBtd]

Continuing my series on CAD AI:
Here Gemini 4 Argon is supposed to build a base for a given DAMIAO 4340 actuator. It uses coding exclusively and finishes after 263 steps with a reward of 0.459 mostly because of missing wire openings

[Pascal Bertrand @PascalBtd]

Starting a new series where I show the latests models' performance on different real-world tasks in Engineering

The first one is GPT-6.1 trying to reconstruct the graph on the right in AutoCAD. It took 181 steps and got a reward of 0.414

[Seldon @seldon_tech]

Podcast episode: https://t.co/8DXqN2FsBT
Career page: https://t.co/t7k6Sc6QCj

[Seldon @seldon_tech]

Our Co-Founders @PascalBtd and @atoof_sh sat down on the Europe on Edge podcast and talked about the current state of multimodal AI and the company's mission. If this mission resonates with you, have a look at our open job listings (links are below).

[Seldon @seldon_tech]

Read more: https://t.co/cuBI1oxOKD

We’re expanding into more real-world engineering workflows across multiple domains, with new tasks and benchmarks coming soon.

[Seldon @seldon_tech]

New SOTA on CADBench: Gemini 3.8 Flash.

30% pass rate on 105 expert-authored CAD tasks in Autodesk Fusion, up from 24.6% for Gemini 3.7 Flash.

Muse Spark 1.3 also improves its mean verifier score by 54% over 1.2, though full task completion remains a challenge.

[Seldon @seldon_tech]

Muse Spark 1.3 improves despite pass rate staying at 0%:
• Mean verifier score: 13.44 → 20.66 (+54%)
• Average runtime: 60.8 → 23.8 minutes (61% shorter)
• Valid tool calls: 76.8% → 93.0%

Better intermediate results and faster runs, but full task completion remains unsolved.

[Pete Soderling @petesoder]

Can computer-use agents use complicated graphical software? You’re watching GPT-5.6 Sol operate Autodesk Fusion for 250 turns as it attempts a real mechanical-design task. The run is part of a benchmark aimed at the question "how good are agents at CAD?"

@seldon_tech's CADBench contains 105 mechanical-design tasks tested across 10 frontier models, and the results show how early reliable, autonomous CAD work remains:

More than two-thirds of the tasks are still unsolved.

That gap matters as agents make their way into the professional tools used to design the physical world. Getting around Autodesk Fusion is one thing, but producing a correct, editable model an engineer can pick up and use is much harder.

CADBench gives us a way to see and measure that frontier.

Great work from the Seldon team! Explore CADBench: https://t.co/npfTr0d4z9

[Seldon @seldon_tech]

@mattsilv Sure! Coming soon

[Seldon @seldon_tech]

How good are agents actually at CAD?

Today we release CADBench, evaluating 10 Models on 105 realistic tasks in Fusion 360

Fable 5 and Gemini 3.7 Flash are #1 with only 24.6% pass rate

[Seldon @seldon_tech]

@CWong24871 Coming soon!

[Seldon @seldon_tech]

@swkkadjo @productflo_io Coming soon!

[Seldon @seldon_tech]

@thinkymachines Inkling struggles a lot with agentic computer use, especially with CAD

[Seldon @seldon_tech]

@samsja19 https://t.co/46M136nt1Y

[Seldon @seldon_tech]

We're hosting another event in San Francisco coming monday, this time a research talk with @samsja19 as guest speaker

Drop by if you're in town!

Sign up link is in the comments

[Seldon @seldon_tech]

Read the full report here:
https://t.co/cuBI1oymAb

[Seldon @seldon_tech]

Gemini 3.7 Flash delivers performance comparable to Fable 5 at a fraction of the cost

We see that many models finish the tasks prematurely, resulting in lower scores and costs

[Seldon @seldon_tech]

@GoogleDeepMind and @Alibaba_Qwen released new models this week. These are the results:

Gemini 3.7 Flash is the new State-of-the-art model on VGI-Bench.

Qwen 3.8 Max comes in as fourth place in after Gemini 3.6 Flash.