Why do we need more benchmarks for frontier models? What you can evaluate and understand, you can optimize. Excited to share some of our work on connecting scientific reasoning, experiments, measurements and process decisions.
@dynamical__ SDL-2 is an open materials science benchmark that follows the whole investigation, from discovering which relationships matter and choosing experiments that distinguish competing explanations to revising an explanation when results surprise and knowing when the evidence is strong enough to act.
It asks agents to infer how much results vary from a few observations and choose experiments that identify which uncertainty would change the decision.
Code, harness, environments and all traces are open!
Can agents use experimental feedback to establish new material capabilities that hardware teams can reliably design around?
Today we’re releasing SDL-2, an open evaluation of how agents choose experiments and decide whether a material process will reliably meet a requirement.
We’re interested in the full process of scientific investigation. That means discovering which relationships matter, choosing experiments that distinguish competing explanations, revising an explanation when results surprise and knowing when the evidence is strong enough to act.
The experiments run in a digital twin of magnet processing, calibrated on measurements from our campaign with Rensselaer Polytechnic Institute (@rpi). Agents investigate unfamiliar powders with an uncertain physics model, a limited experimental budget and results that vary from batch to batch.
Across 32 simulated worlds, frontier models reached correct decisions supported by their own measurements in 44–50% of investigations, compared with 25–28% for other language models and classical experiment design. One in three of their correct answers were unsupported.
Our larger ambition is for materials development to keep pace with hardware design, so teams can design around the capabilities they request, not only those that already exist. SDL-2 makes one part of that measurable, how agents investigate.
We're releasing the evaluation code, harness, digital twin and all 256 investigation traces.
Technical Report: https://t.co/zrSPF2Thll
Code: https://t.co/GDQAYbGTan
Dataset: https://t.co/Jz4DTeLGTr
Look at manufacturing through the lens of software and every part you run teaches you something about the process. Compile enough of that knowledge and the next process gets dramatically faster and cheaper to develop and qualify.
Hardware still has plenty of challenges software doesn't. But we're entering an era that makes application-specific hardware economical for far more teams.
I have enormous respect for anyone crazy enough to invent a new manufacturing process.
Almost everyone will tell you not to. Qualification takes forever. Customers resist change. The economics look terrible until they suddenly don’t.
It’s much easier to build an operationally efficient business serving a market that just needs more capacity with traditional manufacturing.
But here’s what most people miss: Capacity wins you orders fast. A qualified new process can unlock entire markets.
The first takes patience. And once you cross that qualification wall, the floodgates open. Watch what happens in next few months with Machina!
I have enormous respect for anyone crazy enough to invent a new manufacturing process.
Almost everyone will tell you not to. Qualification takes forever. Customers resist change. The economics look terrible until they suddenly don’t.
It’s much easier to build an operationally efficient business serving a market that just needs more capacity with traditional manufacturing.
But here’s what most people miss: Capacity wins you orders fast. A qualified new process can unlock entire markets.
The first takes patience. And once you cross that qualification wall, the floodgates open. Watch what happens in next few months with Machina!
Excited to open up our Lab Notebooks at @dynamical__ !
Kicking off an open science series sharing our work in progress, breaking down which material capabilities could change a hardware design, how we investigate them, and what it would take to make them reliably.
Notebook 001 is our first on electric propulsion systems systems, digging in on understanding if thinner magnets make a propulsion motor lighter.
Much more to come here!
We're opening our Lab Notebook, an open science series sharing our work in progress.
Our notebooks share which material capabilities could change a design, how we investigate them, and what it would take to make them reliably.
Notebook 001 - Can a thinner magnet make a propulsion motor lighter?
Removing magnet material saves rotor mass. Keeping the same torque can mean more current, copper and heat. The hard part is that these constraints are intertwined. Change the magnet and the current, copper, heat and fault response all move with it.
We use a public motor model to explore how the material, winding and motor construction affect that tradeoff.
https://t.co/zkQNGbWawk
We're opening our Lab Notebook, an open science series sharing our work in progress.
Our notebooks share which material capabilities could change a design, how we investigate them, and what it would take to make them reliably.
Notebook 001 - Can a thinner magnet make a propulsion motor lighter?
Removing magnet material saves rotor mass. Keeping the same torque can mean more current, copper and heat. The hard part is that these constraints are intertwined. Change the magnet and the current, copper, heat and fault response all move with it.
We use a public motor model to explore how the material, winding and motor construction affect that tradeoff.
https://t.co/zkQNGbWawk
ChatGPT and Codex are helping engineers build for the physical world.
We’re sharing our progress across mechanical design, electrical engineering, and simulation—from helping our hardware teams iterate faster to testing new workflows with industry collaborators.
If you think building a "software factory" is stolen valor...
Well here is everything our software factory built for our real factory in the last two weeks
- Digital twins of all our manufacturing process
- Our own CAM software for our lasers and press brakes
- Our own nesting engine
- Our own geometry kernel for reading customer parts faster (replacing opencascade)
- In-browser, customer-facing DFM tools (like a bending simulation)
- Custom firmware and hardware on the shop floor automation
- 3D Shipping and packing simulations
- An MCP server so AI agents can quote and order parts
- More performant marketing and documentation pages
- No more saas, every tool we pay for is now gone and we control the code.
This is the smallest our software team has been (1 person, me) and things are only speeding up. We are just getting started!
Today we’re releasing Muse Spark Open, the most capable US open-weight model to date.
Built for coding and agentic work, it excels at long-horizon tasks, complex software engineering, and multi-step reasoning.
Key Model Specs:
→ 816B total / 42B active parameters
→ 1M token context window
→ Natively multimodal across text, image, video, audio
For decades, researchers have sought materials that sort electrons by spin while their magnetism cancels.
In 3 days, 90+ Opus 5.5 agents helped us uncover two room-temperature magnetic semiconductor candidates in simulations: YBaMnFeO₅ and KV[Cr(CN)₆].
KV[Cr(CN)₆] was synthesized back in 1999. Its predicted ability to sort electrons by spin appears to have been hiding in plain sight for 27 years.
Can an LM, starting from random init (!!), learn to generate all of its pretraining data?
Introducing Self-Play Pretraining with Zero Data. Two models start from random initialization: a generator proposes programs for a universal Turing machine and a learner trains on their outputs. We never train on any real data, but see predictable scaling on natural datasets: zero-shot val loss on images, text, audio, and melodies decreases predictably with self-play compute. And the learner develops in-context learning capabilities.
A fun proof-of-concept, co-led with @AdityaCowsik and @KfirDolev and co-authors @gbruno_dl, @ANourya@noahdgoodman, and @YoavLevine.
From a research question to candidate materials.
Xiaomi MiMo-V2.6-Pro worked with Xiaomi’s materials researchers to explore new MOF designs for capturing PFAS — reviewing literature and patents, proposing and checking design hypotheses, and running computational “dry-lab” experiments to identify promising candidates for further validation.
The team estimates a 10× productivity gain, shortening the R&D cycle from one month to 2–3 days.
As Prof. Jinhu Dou of Peking University put it: “In my view, its work on this project—from literature review to materials design and computational evaluation—was on par with that of a well-trained doctoral researcher.”
Read the full case study:
https://t.co/JPanfNyoUC
#XiaomiMiMo #XiaomiMiMoV26 #AIforScience #XiaomiAI
Proud to announce a statewide effort to land the United States Space Academy in Dayton, Ohio.
This is where aviation began. Where two bicycle mechanics left the ground at Huffman Prairie and changed the world. Where Ohioans like Neil Armstrong and John Glenn taught America to look up. Where more than two dozen astronauts have called home. Where America’s first astronauts — the Mercury Seven — were put through their final testing in 1959. To this day, America’s astronauts come to @WrightPattAFB for training at the 711th Human Performance Wing, home of the School of Aerospace Medicine and the military’s only human-rated centrifuge.
Today, Ohio unites to make our case to bring President Trump’s vision for the Space Academy to Wright-Patterson Air Force Base and @wrightstate — the only site with the people, the mission, and the history to secure his legacy.
Thank you to @matthuffman1, @Rob_McColley, @SenJonHusted, @Jim_Jordan, @RepMikeTurner, and @GovMikeDeWine for uniting to make this happen. No politics. Just Ohio. Just America.
From Huffman Prairie to the stars.
In light of the progress in mathematics, we at Edison Scientific and FutureHouse have assembled a set of Millennium Problems for Biology. They are chosen to be very hard to solve but very easy to validate in a simple laboratory environment. Any of these, if solved, would mark a major advance in biotechnology, and most of them would contribute materially towards curing disease. These are, in some sense, the “last reasonable eval” for AI in biology. This was work primarily by @MichaelaThinks and myself, with contributions from many others.
Short descriptions below. The full descriptions of the problems with acceptance criteria are at the Bio Millennium Problems website, linked in the next post. Share more if you have ideas. If they meet our criteria, we’ll add them to our list (with attribution and permission).
Rare earths aren't actually rare, they're as abundant as copper. What's scarce is the ability to purify them, and that's what China controls: over 90% of it.
@maglutheavy founders are borrowing a technique from biotech to modernize rare earth purification. S3 went inside their lab to see their unique take on chromatography.