Can we use Claude Code/Claws living on your terminal/phone to directly assist with our journey towards physical AI?
For physical AI, what interests me is not just VLAs or world models, but also the loop of generating under physical constraints and evaluating under those same constraints.
I recently did some experiments in CAD. One cool thing is that once expressed as a domain-specific language (DSL), it starts to look like constrained code generation and can be built for agents. Physical and geometric constraints can also be clearly rewarded in the loop. So it’s natural to adopt in today’s agents.
Here is a quick demo of a wide-leg jumpsuit generated using this exact DSL approach + Claude Code.
You can turn on the developer mode of your Da Vinci and do Claude code based video editing with its McP.
I’ve also built some other assistant tools and models like motion energy and beat analysis etc. , which should be easily working with vibe editing on Claude if you want some technical bits
Maybe should CLI-ify them and open source my tools
So true. I just granted Claude Code with CLI access of my physical CAD tools and a clear validation instruction and it can make manufacturable shirts and skirts for me.
People still underestimate how agents can be used and it turns out only feedback (reward modeling) and low-cost tool uses (CLIs) matter.
@giffmana Agreed. OSS should be nothing but helping people and making others' lives easier. But people have different philosophies on that, so that's how licenses come into effect.
Here are more results! At end of the day, you can get manufacturable CADs, or prepare simulation-ready assets to train your robotic arms in scale, directly operated from your phone.
We used to run differentiable sim + models on such tasks. This is getting different! More demos:
Can we use Claude Code/Claws living on your terminal/phone to directly assist with our journey towards physical AI?
For physical AI, what interests me is not just VLAs or world models, but also the loop of generating under physical constraints and evaluating under those same constraints.
I recently did some experiments in CAD. One cool thing is that once expressed as a domain-specific language (DSL), it starts to look like constrained code generation and can be built for agents. Physical and geometric constraints can also be clearly rewarded in the loop. So it’s natural to adopt in today’s agents.
Here is a quick demo of a wide-leg jumpsuit generated using this exact DSL approach + Claude Code.
@kaiwynd Yeah that's the production service that we built. For stiffness matching, it depends on if you want to go from the image->stiffness end-to-end pipeline or more real-world/manufacturing side. But either way, I like Huamin's paper a lot (https://t.co/1huiftvZy6).
@kaiwynd Yeah I like PD (the style3D version should already put Chebyshev acceleration there to help) ! We used PD a lot (with ADMM) when we were working on batch garment sims. But we did grind a lot to make sure that the stiffness is well preserved across different fabrics.
@naval Completely agree. This is what I'm doing with vibe-coded projects now: managing all my agents in parallel! It took a couple of days of grinding to get iterations going, but once set up: BOOM!
And it supports mobile end, voice inputs and some self-evolution features.
I'm sharing this in case it’s useful for anyone else diving deep into these topics. It's a living document.
📖 Read the full PDF and check out the repo here: https://t.co/KkV9WgrKC4
Corrections and thoughts are welcome. Feel free to open an issue on GitHub!
I drafted a comprehensive technical guidebook covering knowledge behind today's frontier AI.
It connects the dots between diffusion, flow matching, world models, MoE, multimodal models, reasoning, and more.
Link below:👇
This began as my own learning path.
With my background in math/graphics, I’m familiar with areas like diffusion, 3D, and robotic sim, but I'm actively learning others like reasoning/RL.
I wanted a single reference that connects everything, heavily inspired by @lilianweng’s amazing blog.