Selim Nowicki

Co-founder at @distil_labs
powering AI products with custom small language models

[Selim Nowicki @selimskii]

We’ve just launched mutators at @distil_labs. You tell the system which situations matter for your small model, even the rare ones, and it generates examples of them for training.

https://t.co/ODmCof6cxa

[Selim Nowicki @selimskii]

In our incident-response demo, about 50 of 1,300 tokens are the big model doing the actual thinking. The rest is repetitive domain work. That is where AI feature costs really sit, and it is the part you can move to a small model. New @distil_labs post:

https://t.co/sw84hUb2pf

[Jacek Golebiowski @j_golebiowski]

Most small-model projects die before anyone trains anything. You pick a base model, build a pipeline, find GPUs and write an eval harness, all before you know the task is learnable. We cut that to about 30 minutes. Here's how it works, step by step:

[NVIDIA AI @NVIDIAAI]

https://t.co/hOo42SeZa6

[Suhail @Suhail]

I looked at 13 different providers for even 1 node of B200/B200s while I wait for my order to get delivered.

Zero availability.

I’ve never seen GPU capacity scarcity like this. Prices are also headed towards $6.50-7/gpu/hr. Expect inference to get more expensive.

[Jacek Golebiowski @j_golebiowski]

The open models debate is running on ideology this week. From a production seat it's an empirical question, and for the narrow, high-volume tasks that dominate real AI traffic it looks settled: open weights already won that tier.

[distil labs @distil_labs]

The lowest-scoring of four mid-size MoE models, @NVIDIA's Nemotron-3-Nano, tied for best after fine-tuning, matching a teacher 18x its size. Base rank doesn't predict fine-tuned rank.

[distil labs @distil_labs]

Today we're launching Agent Distillation with @dltHub. Point dlt at your agent traces, get back a hosted model that replaces the expensive agent. Ingest, understand, replace.

[Selim Nowicki @selimskii]

@j_golebiowski MSLMSA?

[Jacek Golebiowski @j_golebiowski]

The fix for expensive LLM agents isn't always replacing them with a smaller model wholesale. It's restructuring the harness so LLMs offload simple subtasks to SLMs.

We cut a bug-fixing agent from $700/day to under $10/day this way. 3,700 orchestrator tokens down to 50.

[Jacek Golebiowski @j_golebiowski]

distil-lfm25-shellper (fine-tuned on @LiquidAI_'s LFM2.5) got 252k downloads last month and we ran no promo.

Either devs love a private local shell assistant, or CLI agents are using it as their natural-language shell tool. Hoping it's the second one.

[Jacek Golebiowski @j_golebiowski]

❗️ Two seats ❗️ just opened up for our #dinner in Berlin this Wednesday.

We're co-hosting with @runpod around the @techeurope_ . A curated group of founders and ML leaders working at the frontier of AI in Europe. If you're running AI in production at scale and want in, DM me

[Jacek Golebiowski @j_golebiowski]

Production traces are not training data. They're domain context for your synthetic data.

We benchmarked the difference across five scenarios — the gap is up to 26 percentage points.

[Liquid AI @liquidai]

Today, we release LFM2.5-350M. Agentic loops at 350M parameters.

A 350M model trained for reliable data extraction and tool use, where models at this scale typically struggle.

<500MB when quantized, built for environments where compute, memory, and latency are constrained.

🧵

[clem 🤗 @ClementDelangue]

After @Pinterest @Airbnb @NotionHQ @cursor_ai, today it’s @eoghan @intercom publicly sharing that they’re finding it better, cheaper, faster to use and train open models themselves rather than use APIs for many tasks.

And hundreds of other companies are doing the same without sharing.

Ultimately, I believe the majority of AI workflows will be in-house based on open-source (vs API). It took much more time than we anticipated but it’s happening now!

[Jacek Golebiowski @j_golebiowski]

Best small language models after fine-tuning (15 models, 9 tasks, rank-based aggregation):

Qwen3-8B — avg rank 2.33 (±0.57)
Qwen3-4B-Instruct-2507 — 3.33 (±1.90)
Llama-3.1-8B — 4.11 (±2.08)
Llama-3.2-3B — 4.11 (±1.28)
Qwen3-1.7B — 4.67 (±1.79)
Qwen3-0.6B — 5.44 (±2.60)

[Jacek Golebiowski @j_golebiowski]

We benchmarked 15 small language models across 9 tasks to find out which one you should actually fine-tune.

The most surprising result: Liquid AI's LFM2-350M ranked #1 for tunability. 350M parameters, absorbing training signal more effectively than models 20x its size.
The entire LFM2 family swept the top 3 spots. No other architecture came close.

LFM2-350M: avg rank 2.11 (±0.89) LFM2-1.2B: avg rank 3.44 LFM2.5-1.2B-Instruct: avg rank 4.89
That tight CI means it's consistent across every task type, not just a few lucky benchmarks.

[Santiago @svpino]

Fine-tuning a model with just a prompt sounds like a joke until you try it.

Prompt engineering with a general-purpose model can only get you so far. Prompt engineering influences how a model uses its knowledge, but it does not introduce new knowledge into the mix.

If you want complete control over the results of your model, you need fine-tuning.

But fine-tuning is hard:

• You need a curated dataset (hard)
• You need distributed training pipelines (hard + expensive)
• You need a lot of compute (hard)

Fine-tuning takes time, money, and skill. Most companies have neither of these.

Here is where the idea of vibe-tuning comes in.

Vibe-tuning is a method for fine-tuning a small language model using only a natural language prompt.

You describe what you want, and the tuner generates synthetic data, sets up distillation, fine-tunes the model, and evaluates the results.

The first time I heard about this was from DistilLabs. They are currently automating the entire fine-tuning process:

1. You provide a prompt describing the task
2.The platform generates and labels synthetic training data
3. You pick a Teacher model (say gpt-oss-120b) and a Student model (say llama-3.2-3B)
4. The platform distills, fine-tunes, benchmarks, and delivers a downloadable small language model
5. You can deploy this model and start using it right away.

The technique builds on model distillation: transferring knowledge from a large "teacher" model to a compact "student" model that's cheaper and faster.

Honestly, this is huge.

You can literally teach a model your company's tone, classification rules, or tool-calling logic by writing a few sentences in English.

[Jacek Golebiowski @j_golebiowski]

We trained a family of small language models (SLMs) tuned for policy-aware PII redaction. The 1B model, which can be deployed locally, matches a frontier 600B+ LLM model (DeepSeek 3.1) in prediction accuracy. Check it out at https://t.co/1R7fBOfAol

Let me know what do you think.

[Andrej Karpathy @karpathy]

Something I am experimenting with. I copy pasted:

1) the full podcast transcript
2) the bitter lesson blog post
3) my full post above

To ChatGPT. The interesting part is you can fork the conversation context to ask any questions and take it in whatever direction with chat:
https://t.co/m98ivfic67