Leonard Salewski

Post-training LLMs at @Aleph__Alpha | Ex intern at @Nvidia and @Bosch_AI

[Leonard Salewski @L_Salewski]

📢 #scholarGPT now can perform complex filtering operations like selecting a specific year 📅, making research even more efficient! 💪 Build with self-query from @langchain
Sign up on the waitlist to be the first to try it out https://t.co/iRS47YelE4! 🚀✨ #ChatGPT #Research🧑‍💻

[Leonard Salewski @L_Salewski]

One fun aspect of our latest blog post: You can change its reasoning effort.

Drag the slider and the post rewrites itself word by word, from a 12-minute read at high down to 2 minutes at none.

[Aleph Alpha @Aleph__Alpha]

Kolibri can reason at four efforts: none, low, medium, and high. Our SFT training traces didn't have effort labels, so we had to label them post-hoc.

A new blog post by @L_Salewski explains how we did this, with interactive figures 🧵

[Aleph Alpha @Aleph__Alpha]

Parsy is currently the best parsing model in the world.

Read on for how we built this model on top of our parsing harness. 🧵

[Leonard Salewski @L_Salewski]

@infornomics @Aleph__Alpha @IlhanScheer Yes, our model factory really supports us to go as quickly as possible! We describe our SFT safety and alignment work in Section 3.1.1.5 (page 56). The last paragraph of Section 3.2.2.3 also gives a few insights about the safety environments that we used during RL (page 77).

[Leonard Salewski @L_Salewski]

The Kolibri tech report is now on arXiv: https://t.co/WRxBkvnsry. If you haven't read it yet, give it a read - it covers all steps on how to build a modern LLM.

[Leonard Salewski @L_Salewski]

@jvr0x @Aleph__Alpha If you want to know what we did for the different reasoning efforts, I explained some of our work here https://t.co/4VktExXnZE

[Leonard Salewski @L_Salewski]

Almost all models today have reasoning effort, here is how we built it for Kolibri. Our model supports four efforts: none, low, medium, high.

However, our SFT data had no effort labels. Here is how we got them anyway, using only existing data 🧵

[Leonard Salewski @L_Salewski]

@jvr0x @Aleph__Alpha Thanks for the feedback and giving Kolibri a spin. We'll keep this in mind when building the next iteration!

[Leonard Salewski @L_Salewski]

@humansandaiboss @SKatalystAI Thanks for trying Kolibri and testing it on rust! We included a lot of code in our training data and SWE Environments with multiple harnesses in our RL training to make it as good as possible.

[Leonard Salewski @L_Salewski]

@arcencilo @Aleph__Alpha Yes, we integrated German throughout all parts for the model pipeline. From pre-training data curation, to post-training, to dedicated German evals. You can read all about it in our tech report: https://t.co/ddPM46CoZ0

[Leonard Salewski @L_Salewski]

@itsolelehmann Awesome that you like Kolibri so much! We at @Aleph__Alpha are certainly excited for what's to come!

[Leonard Salewski @L_Salewski]

@irombie We really tried to put a lot of details into the report, and also included lessons learned throughout all parts of the model training pipeline. Happy to answer any questions!

[Murali Kondragunta @gitlostmurali]

During its development, Kolibri surprised us in ways we didn't expect with reward hacking & escaping sandboxes. We were a little proud but also increased our sandbox isolation

For an 80B-A3B model, it is unexpected. All the details are in "Lessons learned" section in tech report

[Aleph Alpha @Aleph__Alpha]

We started the year with the conviction that Europe needs its own sovereign AI. Built in Germany from scratch and on infrastructure in Europe specifically for our customers.

This weekend, we released Kolibri.

Meet the team behind it 🧵

[Felix Berkenkamp @F_Berkenkamp]

My favorite reinforcement learning plot from our Kolibri report: Inference v.s. training log probs, background is the logit gradient norm of importance-sampled Reinforce. Numerically unstable for small inference log-probs, but you can do exact token-level gradient clipping in RL.

[Leonard Salewski @L_Salewski]

@osanseviero @Aleph__Alpha just released Kolibri, a 78B total, 3.46B active open model https://t.co/BuE29D4hRN, alongside a 189 page tech report https://t.co/ddPM46CoZ0

[Aleph Alpha @Aleph__Alpha]

Small bird, fast wings, Kolibri is here.

78B parameters. 3.46B active. Up to 1M tokens of context. Built in Europe.

Now the weights are yours. Run it on your own hardware, under Apache 2.0.

[Leonard Salewski @L_Salewski]

@Ahmadbukhari @bta_ckr @Aleph__Alpha We mostly used GLM 5.2/5.3 and Qwen3.8-27B (see Section 3.1.1 of our tech report for more details!), However, we didn't simply take these traces, but carefully curated them, filtering for biases, wrong samples, etc. (see also Section 3.1.1.5).
https://t.co/ddPM46CWOy

[Leonard Salewski @L_Salewski]

@HenkPoley @aroogle @Aleph__Alpha Kolibri was post-trained as well and actually performs quite well on AIME 2025 and AIME 2026. Check out page 100 of our tech report (https://t.co/ddPM46CoZ0) with all the full post-training evals, including Math, Code, Agentic and Long-context.

[Leonard Salewski @L_Salewski]

@LuisCor80246896 @aidangomez Kolibri is open weight, you can download it here: https://t.co/HJ5569rbjw

[Leonard Salewski @L_Salewski]

@aidangomez We are all looking forward to build something even bigger together, thanks for the support!

[Leonard Salewski @L_Salewski]

Check section 3.1.4.1 of our tech report (https://t.co/ddPM46CoZ0) for more details. It also has many more details, so certainly worth a read on how to train a model end-2-end!

[Leonard Salewski @L_Salewski]

Here are two neat speed tricks we found while SFT'ing Kolibri 🧵

1: We use standard Next-k-fit online bin packing with document masking maximises useful tokens per sequence. Whats easy to miss: less padding means a larger effective batchsize, so the learning rate can go up.

[Aleph Alpha @Aleph__Alpha]

In post-training, we taught Kolibri to think and answer in both English and German consistently across SFT and RL alike. We generated large amounts of synthetic data to control how Kolibri answers and trained with RL on more than 1.2 million tasks, including ones built around our customer needs, and ones to teach it to say “I don’t know” when the context is insufficient.

[Leonard Salewski @L_Salewski]

This adds a small bias: a step's microbatches are no longer a plain random draw. In our runs the wall-clock gain won by a wide margin: median throughput per GPU went from 8,610 to 15,850 tokens/s on 64 B300s, +84%, same data, model and kernels.