Devin White

AI Research Scientist @thewebAI | Reinforcement learning, RLHF & LLMs

[Devin White @DevinWhiteAI]

I’m excited to share that I’ve joined webAI’s Intelligence Lab as an AI Research Scientist!

I’m incredibly grateful to everyone who has supported me, collaborated with me, and helped me get to this point. Looking forward to what the future holds!

[Devin White @DevinWhiteAI]

@ManlingLi_ This is awesome! Really cool work

[ali @aliteracy]

introducing the modern foundation model training stack (2026):
- pre-pre-training
- pre-training
- post-pre-training
- mid-training
- pre-post-training
- post-training
- post-post-training(?)

[Devin White @DevinWhiteAI]

Joining WIL has been exciting. Working with this team and pushing the limits of reasoning on the edge has been a lot of fun. Cutter put it best in the lab: “We can learn anything we put our minds to.” Super excited for what the future holds!

[webAI @thewebAI]

How does a 3B parameter model compete with frontier models?

In our latest video, our Intelligence Lab team walks through how they built the TwIL reasoning models. The thesis behind formal reasoning, the moment that reshaped their data pipeline, and where the lab is headed next.

Watch the full video: https://t.co/A7Cahkwx1J

[Devin White @DevinWhiteAI]

@ManlingLi_ @xdotli This is really cool!

[Devin White @DevinWhiteAI]

We’ve been playing around with the new TwIL-LM3-Pro model and in this maze problem it solved it so fast and even beat Qwen3.8-27B!

Seeing this model perform this good at only ~4B is crazy, definitely should check it out if you haven’t already!

[webAI @thewebAI]

We ran TwIL on a CPU with 15 GiB of RAM. No GPU. It generated tokens 98x faster than Qwen3.8-27B on the same hardware.

The test:
- Intel Xeon CPU, 4 threads
- 15 GiB RAM
- No GPU

Results:
- TwIL-LM3-Pro (2.24 GB weights): 10.47 tokens per second
- Qwen3.5-9B (5.67 GB weights): 4.36 tokens per second
- Qwen3.8-27B (16.45 GB weights): 0.107 tokens per second

TwIL and Qwen 9B both run at the RAM's memory limit of about 24 GB/s. Qwen 9B is 2.4x slower only because its weights are 2.5x larger.

Qwen 27B's 16.45 GB of weights don't fit in the 15 GiB of RAM. Parts have to be re-read from disk for every token. Size alone would make it 7.3x slower. The disk reads add another 13x. Total: 98x slower than TwIL.

With enough RAM, the gap would shrink to about 7x. But most people don't have unlimited RAM, and most people don't have a top-tier GPU. The model you can actually fit matters more than the one you can't.

Download TwIL-LM3-Pro on @huggingface: https://t.co/pTRuIDyV0L

[Devin White @DevinWhiteAI]

Some really cool work coming out of the lab

[webAI @thewebAI]

Base-7 arithmetic is a brutal test for small models.

There's no base-10 shortcut, so every carry has to be reasoned through directly in the target base. Most 3B models either skip the carry logic or hallucinate the conversion back to base-10 at the end.

TwIL-LM3-Pro self-corrects a wrong carry mid-trace inside <think>. 3.6B parameters, running locally.

Final: 114₇ = 60₁₀.

Try it on @huggingface: https://t.co/JyiFTWJySy

[webAI @thewebAI]

Base-7 arithmetic is a brutal test for small models.

There's no base-10 shortcut, so every carry has to be reasoned through directly in the target base. Most 3B models either skip the carry logic or hallucinate the conversion back to base-10 at the end.

TwIL-LM3-Pro self-corrects a wrong carry mid-trace inside <think>. 3.6B parameters, running locally.

Final: 114₇ = 60₁₀.

Try it on @huggingface: https://t.co/JyiFTWJySy

[webAI @thewebAI]

LLMs are getting more rambly. We built TwIL-LM3-Pro to go the other way.

Less than half the tokens of VibeThinker-3B and Qwen3-8B. 1.7x and 5.4x faster answers on general reasoning benchmarks.

Full benchmarks: https://t.co/dr1LDchxMr

[David Stout @Davidstout]

Half a million downloads in a month. Today, our open source family takes another step forward.

Thank you for the incredible support behind our first-generation models. We’re excited to introduce TwIL-LM3-Pro.

At just 3.6 billion parameters, it brings powerful reasoning to everyday computers, with quantized builds that run locally. No cloud required.

In our evaluation:
Formal logic: Highest recorded headline score among the small models compared—beating China’s VibeThinker-3B by 35% and Qwen3.5-4B by 24%, and Liquid AI’s LFM2.5-8B-A1B by 47%.

Broader reasoning: 95% on SVAMP and 64.1% on MuSR, the highest recorded scores among the small models compared.

BIG-Bench Hard’s logic subset: 95.4%, compared with VibeThinker-3B’s 61.1%.

We believe AI is entering a post-training era. The advantage will increasingly belong to companies with the best pipelines and those that can produce capable, personalized intelligence faster and more efficiently, then put it on devices people already own.

That’s what we’re building at webAI. And we’re only beginning to share what’s coming out of our lab.

Coming soon: Meridian, our family of frontier-class models built to run on device. Our most advanced models will be available through the @thewebAI application.

Join the waitlist as we expand access. Proudly built in Austin, Texas. 🇺🇸

[Tim Cook @tim_cook]

Sending lots of love to the Apple community on my last day as CEO. My title changes tomorrow, but the love I have for the Apple community never will. Thank you for being a constant source of inspiration. My gratitude is endless, and I’m excited for the next chapter!

[Surya @kickingkeys]

You can just RL a coding model to paint with javascript btw

[Ishan Ahluwalia @Ishan_Ahlu]

When your model generalizes 😂

[Manling Li @ManlingLi_]

Kellogg is hiring, with a focus on AI!

Application reviewing will be started in July.👇

[Tom Yeh @ProfTomYeh]

Self Attention vs Cross Attention by hand ✍️ Resize the matrices yourself 👉 https://t.co/O6XAl7TdcL

Two attention mechanisms, side by side. Both project X into queries; both compute attention via S = Kᵀ × Q and F = V × A. The only difference is the source of K and V.

Self attention uses X for everything. Q, K, and V all come from projecting X. Each X token attends to every other X token. The score matrix S is square — 128 × 128.

Cross attention uses X for queries and a second sequence E for keys and values. Each X token attends to every E token instead. The score matrix S is rectangular — 64 × 128.

Notice what's shared and what's not:

X is the same in both — same 36 × 128 input.

Q and K share the 16 dimension — that's what makes the dot product Kᵀ × Q valid in either case.

V dimensions are independent: self-attention uses 12, cross-attention uses 12. The choice doesn't depend on which mechanism you're using; it depends on what output dimension your downstream layer expects.

[Théo Vincent @Theo_Vincent_]

Working on AutoRL is essential for making reinforcement learning more scalable and used in the industry.

Excited to be part of this workshop!🔥

[Devin White @DevinWhiteAI]

@wesroth If true this could be huge for research!

[Devin White @DevinWhiteAI]

@googledevs Made a simple RLHF pipeline for Gemma 3 fine-tuning.

Includes preprocessing, reward model training and RLOO fine-tuning!

Code: https://t.co/YKr43Iv2le

#RLHF #Gemma #AI #GoogleDev (Image generated by @NanoBanana)

[Devin White @DevinWhiteAI]

@NewsFromGoogle This is great! With how good Gemini 3 Pro is, I'm excited to see what updates Siri gets!

[Devin White @DevinWhiteAI]

Full Code: https://t.co/YKr43Iv2le

[Devin White @DevinWhiteAI]

Ever wanted to learn the basics of Fine-tuning a LLM?

I just built a complete, single GPU friendly, end to end pipeline for RLHF fine-tuning using @huggingface TRL!

Here is the 2 stage process I did:
1⃣ Train a "Judge" (reward model) on human preferences (a subset of the @NVIDIAAI HelpSteer3 dataset)
2⃣ Align @GoogleDeepMind Gemma 3 with the judge using RLOO!

Try out the code below👇

#RLHF #ML #AI (Image generated by @NanoBanana )