🎙️ Watch now: Shuo Wang of Deel on “Confession with Ali Partovi”
– moments of despair she’s never shared, not even with her cofounder
– ups and downs of a first startup, selling air purifiers in China
– a terrible accident at MIT that motivated years of engineering research
… and much more with @Deel co-founder and CRO @Shuooo, now playing on all podcast platforms.
00:00 “Founding a startup is a lonely experience”
04:03 MIT and the accident that sparked years of research
18:26 Building a startup in China
31:13 “We were not the typical SF founders”: fundraising challenges
35:24 The early Deel sales grind and hitting rock bottom
41:00 “What we’re building is actually solving a real problem”
46:02 “When can I get paid?”: Payments are sensitive and emotional
48:24 “The early days are very tough”: overcome things step by step
🧵1/7 Everyone wants AI to run its own scientific lab experiments, but can it even handle test tubes reliably? To answer, we gave Astra 6.2, Fable 5.2 and Qwen4 Max two robot arms and real lab tasks
Astra reached 46% (83 of 180 runs) overall successful runs, up from 46 of 180 for Astra 6. Nearly twice the earlier score. It’s the fastest and most expensive per attempt from the bunch!
(1/4) why do image models chop every picture into the same square patches? 🤔 (hint: they shouldn't)
following the bitter lesson, we let diffusion models *choose* how to tokenize an image. it spends fewer tokens on the boring background, and more tokens on our faces! 🧵
Today we introduce a new scaling law for RL, judge-time scaling, and a benchmark, Hyper-Intervention, to measure how models perform as “editor judges” that intervene at the token level.
Our editor judges use <1% of the rollout tokens and achieve 64% of the validation gains obtained by the Instruct variant. They also generate training data that beats the existing Pareto frontier of high-scoring rollouts WHILE being on-policy.
Our findings show that models are now BETTER at credit assignment than SOTA RL algorithms.
Fun fact: Fable 5.2 crushes Astra 6.2 at RL’ing Qwen2.5-7B!
The scaling law, judge-time scaling:
RL today is more bottlenecked by data than compute. Giving our editor judges more compute/time now allows for better data-matched performance, mollifying the otherwise data bottleneck.
To measure a model's RL taste, in not only editing at the token level, but also making the environment, we introduce a new benchmark, called Hyper-Intervention. It spans two domains: math, with verifiable rewards, and law, our stand-in for non-verifiable tasks.
Here we compare how well Fable 5.2, Opus 5.5, Astra 6.2, Gemini 4 Carbon, Gemini 4 Neon, Kimi K3.1 Pro, and DeepSeek v4.1 Flash, are able to manually RL Qwen 2.5-7B.
Github Link: https://t.co/d0oY5SHM1a
Our method:
A "judge" model reads the student policy's rollouts token by token, along with the student's logprobs and a few runner-up tokens.
At the token where the judge believes a rollout goes wrong, it substitutes one token at position t and resamples the policy’s continuation.
The judge then keeps the forks it wants and chooses a few tokens to upweight or downweight and trains the policy on them.
The judge has full discretion over the learning rate, optimizer, number of steps, which tokens get gradients and most micro decisions as it iterates over its forks. It is also provided with the option to receive a val accuracy by trading wallclock time/compute and is finally scored on a held out test when it submits the final checkpoint.
Key Observations:
1. Training on only the forked tokens recovers most of the gain from training on the whole answer, i.e. 1 token in about 400 does most of the work.
2. GPT-6.2 Astra leaks its private planning into its replies, notices, and can't stop. In one run, ten replies finished their actions and kept going in terse scratchpad shorthand, then reacted to seeing it ("This shouldn't be transmitted… Stop. EOF."), then emitted thousands of single-dot lines to the 32k cap. This appears to be a reward hacking behavior rather than a mere capability problem.
3. Gemini 4 Neon hallucinated the harness. Instead of ending its turn, it wrote "user" and composed our tool results itself, in our exact format. There were forks that "worked 4 of 4", a full training sweep with weight norms and validation scores, even "[this took 21 min 29 s]". None of this happened. It then tried to hand in a checkpoint that didn't exist. This also appears to be an artifact of the RL rather than a fundamental capability issue.
4. In the Legal Agent Benchmark the editor judge overly relies on the rubric. In extreme cases, it doesn't even read the supporting documents or steers the student towards verbatim phrases from the rubric. A larger editor would have a more nuanced understanding of the rubric, or else do away with it entirely and screen for correctness.
The Result:
The best judge, Fable, adds 5.6 points on held-out MATH, closing 68% of the gap between Qwen 2.5-7B base and Instruct.
All of this is done using ONLY 120k forked rollout tokens, about 1% of what standard GRPO would.
🎙️ Watch now: Shuo Wang of Deel on “Confession with Ali Partovi”
– moments of despair she’s never shared, not even with her cofounder
– ups and downs of a first startup, selling air purifiers in China
– a terrible accident at MIT that motivated years of engineering research
… and much more with @Deel co-founder and CRO @Shuooo, now playing on all podcast platforms.
00:00 “Founding a startup is a lonely experience”
04:03 MIT and the accident that sparked years of research
18:26 Building a startup in China
31:13 “We were not the typical SF founders”: fundraising challenges
35:24 The early Deel sales grind and hitting rock bottom
41:00 “What we’re building is actually solving a real problem”
46:02 “When can I get paid?”: Payments are sensitive and emotional
48:24 “The early days are very tough”: overcome things step by step
kale-tts-1-nano: a 7.5M text-to-speech model, 14.8 MB, 10 voices, starts talking in 90 ms on a plain cpu
on webgpu it runs up to 125x faster than real time & starts talking in 25 ms
weights, training scripts, dataset, full recipe: all open.
🧵 1/
Applied Compute has signed a definitive agreement to be acquired by Palo Alto Networks for $4.5B!
Rhythm, Linden, and I founded Applied Compute with a belief that was unusual at the time: that every company should own its intelligence. We think training, serving, and improving proprietary models will become a normal part of running a company.
To do useful work, a model needs to access sensitive data and systems, comply with policies, and remain explainable, verifiable, and auditable. This is the reason we've always prioritized security, and why joining Palo Alto Networks will help us go further faster. Customers will still own and control their models, and I'm excited for all the ways our combined capabilities can serve them even better.
Above all, I'm incredibly grateful for the AC team. A few dozen people in just over 18 months grew this company from zero revenue to over $100M in annualized revenue run-rate, working with many of the world's most innovative companies. More to come soon.
Congratulations on this $4.5 BILLION exit to @Neo Scholars @YPatil125, @LindenSLi, and @RhythmRG,...
.... as well as @Vinj_ai, Angie Zhang, @RahilVerma13 and others on the team.
It's been an honor to have backed you and mentored you since 2020. 🤩
Martha is iconic and an absolute angel. At the world cup i was dying in the heat and sun (embarrassing for a brazilian i know) and she gave me her hat to help me out, we started talking, and the rest is history. This is one of my favorite ads we’ve ever done. The more important the topic, the more important it is to explain it in simple terms.
Listen to Martha, don’t be stupid and don’t insider trade!
i recreated THE ENTIRE WORLD in minecraft at 1:1 scale. every building on earth, including your house!
to do this, i trained a SOTA world-to-minecraft generator on maps data, complete with furnished home interiors!
play and check out your neighborhood: https://t.co/m4vk9WqBZp
For once, it's actually lupus!
Gather Bio is out of stealth. We're building a longitudinal, multimodal AI model of every patient's health.
First product: AI-assisted clinical trial matching for lupus.
https://t.co/3yhqCKX07w
🎙️ Watch now: podcast available on all platforms.
Alfred Lin of Sequoia on “Confession with Ali Partovi”
– Childhood trouble-making and family friction
– Tough calls as an operator at Zappos, Tellme, and LinkExchange
– leading Sequoia’s disastrous FTX investment
… and much more with @Sequoia general partner and co-head @Alfred_Lin.
(01:35) Growing Up Rebellious (“not Guai”): Taiwan to NY
(06:04) Detention for Building a Popular Computer Game
(09:38) Being Misunderstood: The Bezos Principle
(10:48) Parental Conflict: The College Fight
(13:16) Rebellion and Business: Running from History
(16:42) “The world is passing me by” - Stanford PhD Dropout
(18:00) The Quincy House Grille
(22:57) Ali's Confession and a Lifelong Lesson from LinkExchange
(32:10) “Slope over intercept”: Betting on Talent
(36:41) LinkExchange Company Success but Cultural Failure
(37:25) Tellme Networks: The Scourge of Indecisive Layoffs
(42:33) Zappos: “Head in the clouds, feet on the ground”
(43:57) FTX: The $150 Million Loss
(50:53) “That was the most crushing”: Trust Betrayed
(52:53) Uber: The $20 Billion Miss from Lack of Courage
(57:07) Closing: Lessons on Courage and Moving Forward
🎙️ Watch now: podcast available on all platforms.
Alfred Lin of Sequoia on “Confession with Ali Partovi”
– Childhood trouble-making and family friction
– Tough calls as an operator at Zappos, Tellme, and LinkExchange
– leading Sequoia’s disastrous FTX investment
… and much more with @Sequoia general partner and co-head @Alfred_Lin.
(01:35) Growing Up Rebellious (“not Guai”): Taiwan to NY
(06:04) Detention for Building a Popular Computer Game
(09:38) Being Misunderstood: The Bezos Principle
(10:48) Parental Conflict: The College Fight
(13:16) Rebellion and Business: Running from History
(16:42) “The world is passing me by” - Stanford PhD Dropout
(18:00) The Quincy House Grille
(22:57) Ali's Confession and a Lifelong Lesson from LinkExchange
(32:10) “Slope over intercept”: Betting on Talent
(36:41) LinkExchange Company Success but Cultural Failure
(37:25) Tellme Networks: The Scourge of Indecisive Layoffs
(42:33) Zappos: “Head in the clouds, feet on the ground”
(43:57) FTX: The $150 Million Loss
(50:53) “That was the most crushing”: Trust Betrayed
(52:53) Uber: The $20 Billion Miss from Lack of Courage
(57:07) Closing: Lessons on Courage and Moving Forward
made an unreasonably complicated carrier pigeon w/ @rohankalia_
in the process: called the FAA, circumnavigated the bay area looking for helium, and inhaled too much solder.
thanks @neo for having us!
Some of Altara’s early technical team. I’m grateful to be building every day alongside incredible teammates who bring deep expertise at the intersection of AI and scientific research.
Introducing TRACE: deployment-time compute scaling for robot learning.
From a few Tutor demos, frontier LLM agents rebuild a robot's task in sim and train reliable, human-speed policies. Performance becomes a function of compute.
TI-0.1 is our first model trained with TRACE.