Sebastian Raschka

ML/AI research engineer. Ex stats professor.
Author of "Build a Large Language Model From Scratch" ( & reasoning (

[Sebastian Raschka @rasbt]

I put together a mega write-up on GPT-6 Astra & looped transformers.

How looped transformers / recurrent depth works, cost-tradeoffs, whether it hides reasoning traces, with lots of figures and a tour of recent looped transformer research.

[Sebastian Raschka @rasbt]

The complete reasoning-from-scratch video series is now up (and linked) here: https://t.co/5IOHLTCIGt

(Taking a short break but will be back with LLM model and research updates in November!)

[Sebastian Raschka @rasbt]

@fireplyai That would be an interesting follow-up experiment. I.e., training the model on the same dataset with the traces hidden/removed.

[Sebastian Raschka @rasbt]

Reasoning From Scratch round 8, the finale!
This last lecture in the series covers distillation as an efficient alternative to the previous reinforcement learning training runs.

(Planning to take a brief break, but I will be back!)

00:00 Introduction and series recap
01:07 Why distill a reasoning model?
02:51 Soft vs. hard distillation
11:06 On-policy vs. off-policy distillation
18:09 Preparing data for reasoning distillation
22:35 Loading the DeepSeek R1 MATH dataset
28:04 Inspecting reasoning traces and final answers
31:27 Tokenization and chat templates
35:58 Pre-tokenizing the training dataset
39:01 Sequence lengths and the 2,048-token limit
42:38 Creating training and validation splits
45:32 Loading the Qwen3-0.6B base model
46:24 Computing answer-token log probabilities
54:10 Cross-entropy loss for distillation
57:25 Training and validation loss helpers
1:02:03 Implementing the distillation training loop
1:07:26 Running the standalone training script
1:09:41 Plotting training and validation losses
1:11:31 Evaluating checkpoints on MATH-500
1:14:09 Comparing teachers and distillation results
1:16:34 Future directions for reasoning models
1:19:41 Bonus exercises and next steps

[Sebastian Raschka @rasbt]

@yusufksadek The student was around 44%. Yeah, one could filter by correctness. I haven't done this here because in real-world use cases we usually don't know what's the correct answer in many cases.

[Sebastian Raschka @rasbt]

@ArgusPlatformAI On the MATH (eval) dataset the tiny 0.6B student got 44% accuracy after distillation (up from 15% for the base model). The 0.6B reasoning reference model got around 48%.

The 671B teacher model got 91.2%. But yeah, it's also 1000x larger.

[Sebastian Raschka @rasbt]

@BreezeOg1 Thanks!

[Sebastian Raschka @rasbt]

@geminitfwang You mean in the training set? About 44%.

[Sebastian Raschka @rasbt]

@Shuyin_ben Glad you are liking the book (and videos).
Regarding the article you linked, it looks interesting. I haven't had a chance to read it carefully and fully, but it looks like it summarizes several known ideas in one place, which is nice.

[Sebastian Raschka @rasbt]

@randomdb_ @BenjDicken ha that's hilarious (sry, I am not managing the amazon listing, that's the publisher :D)

[Sebastian Raschka @rasbt]

@krystof_mitka Glad you liked it overall, and thanks for the feedback!

[Sebastian Raschka @rasbt]

And one more time, the YouTube link: https://t.co/w5YwuYILWN

[Sebastian Raschka @rasbt]

@MLaurusevicius Yes, even DeepSeek V3.2 (as mentioned in the paper) left it out; there are several others

[Sebastian Raschka @rasbt]

@MLaurusevicius Not necessarily but it adds more complexity and is not necessary for some domains (like math)

[Sebastian Raschka @rasbt]

@md_usman73 Yes, and I still stand by that

[Sebastian Raschka @rasbt]

@sroyyors Thanks!

[Sebastian Raschka @rasbt]

@BiploveYadav The model (training) should work fine without format reward, it’s just an optional stylistic thing. If the model never gets the answers right then you’d need to start with a simpler dataset.

[Sebastian Raschka @rasbt]

Reasoning From Scratch: Reinforcement Learning with Verifiable Rewards (RLVR) round 2.
Covering clipped policy ratios, KL loss term, format rewards, and other GRPO tips & tricks.

00:00 Introduction and recap
01:52 Interpreting basic GRPO training metrics
06:34 Planned improvements to GRPO
08:58 Running longer training jobs with Python scripts
13:39 Running the baseline GRPO training script
17:29 Loading and plotting training logs
19:29 Diagnosing unstable training
23:55 Evaluating checkpoints on MATH-500
26:26 Downloading existing checkpoints
30:09 Tracking advantage statistics
34:53 Understanding entropy
40:32 Computing entropy in PyTorch
44:17 Interpreting entropy values
48:58 Adding entropy tracking to GRPO
53:36 Analyzing advantage and entropy metrics
56:18 Stabilizing GRPO with clipped policy ratios
1:03:27 Implementing the clipped policy loss
1:09:39 Analyzing clipped policy training results
1:11:25 KL divergence and reward hacking
1:15:12 Adding a KL loss term
1:20:34 Limitations of the simplified KL loss
1:23:04 Format rewards and think tags
1:25:47 Adding special tokens to the tokenizer
1:30:29 Implementing the format reward
1:35:56 Analyzing format reward training
1:38:25 Rewarding format only for correct answers
1:40:48 Further GRPO improvements from research
1:45:43 Next steps and distillation

[Sebastian Raschka @rasbt]

@acatovicx Thanks :)

[Sebastian Raschka @rasbt]

@sachintwtss Glad this was useful!

[Sebastian Raschka @rasbt]

@ManuAF6 Clipped policy ratios were the ones that that helped most with that in this case