spent a while rebuilding diffusion models from scratch, from 1000-step DDPM to 1-step generators
wrote up what I learned https://t.co/aKwWkxfbsu
make the progression easy to follow in code, without hiding the interesting parts inside large frameworks
https://t.co/1HgNEtjX49
Today we are releasing JST-2, the world's first full-stack physics-native world foundation model.
Many world models learn in pixel/video space. We went back to first principles and built the entire model stack in 3D.
Generate a 3D scene in seconds and expand it in real time. Create characters with motion built in, all through one general architecture.
been playing around with worlds from Viggle JST2 and characters + motions from PINOC. turned it into a little game prototype where you can rift-jump between 6 3D worlds without a cut.
a bit of Ratchet & Clank inspiration :)
here’s a clip
Today we are releasing JST-2, the world's first full-stack physics-native world foundation model.
Many world models learn in pixel/video space. We went back to first principles and built the entire model stack in 3D.
Generate a 3D scene in seconds and expand it in real time. Create characters with motion built in, all through one general architecture.
shower thought:
we're in a scaling phase now, but ai progress is a cycle of scaling and reinvention.
a good architecture can keep improving as we scale up data and params but as gains slow down, finding a better arch becomes more rewarding than scaling the old one.
a very odd loss curve you’ve probably seen: loss creeps up within each epoch, then drops at the next.
I explain this “Epochal Sawtooth Phenomenon” from a new angle, simple but principled: loss tracks the expected gap since a sample was last trained on.
https://t.co/jJr76JTBwa
The agents proposed 397 experiments, each with a design, independent review, correctness gates, then timing against the best build. 98 produced improvement and 289 retired with a written reason. Every outcome went into a shared record the next proposal reads first.
3/3 two ideas
a. richer personalized memory, a map of what i know and dont know, not just who i am or what i prefer.
b. maybe a rpg-style quest trees design so i can pause and fork side quests to fill the knowledge gap then retrun to the main quest.
1/3 talking to agent sometimes feels like knowledge yak shaving. like looking up a word in an English-only dictionary, only to find more words i dont know. or going down a wikipedia rabbit hole until i forget what i was looking for.
I built nano_diffusion to understand the journey.
implementing DDPM, Rectified Flow, Consistency Models, MeanFlow, and DMD2 from scratch in PyTorch.
make that progression easy to follow in code, without hiding the interesting parts inside a large framework.