Daniel van Strien

Machine Learning Librarian @huggingface 🤗
I like big datasets and small models.

[Daniel van Strien @vanstriendaniel]

Uploaded a dataset of 115,293 illustrated pages from the Encyclopaedia Britannica, 1st edition (1768) to 14th (1929) to the Hub

https://t.co/xrVwV8BNyE

[Daniel van Strien @vanstriendaniel]

@AceEngineer1 @huggingface @LightOnIO @vllm_project https://t.co/RJbpEggNBV

[Daniel van Strien @vanstriendaniel]

OCR models are getting very fast!

I OCR'd 14,483 pages of fossil fuel industry documents with LightOnOCR-3-0.8B from @LightOnIO. One RTX PRO 6000 on @huggingface Jobs with @vllm_project, about 7.7 pages/s once running. $1.67 in total, about $0.12 per 1,000 pages.

I tuned the recipe for Jobs. The video is the run in progress, with each page appearing when its text comes back.

One command to run it on your own pages: https://t.co/0hnnKte1GQ

[Daniel van Strien @vanstriendaniel]

OCR models are getting very fast!

I OCR'd 14,483 pages of fossil fuel industry documents with LightOnOCR-3-0.8B from @LightOnIO. One RTX PRO 6000 on @huggingface Jobs with @vllm_project, about 7.7 pages/s once running. $1.67 in total, about $0.12 per 1,000 pages.

I tuned the recipe for Jobs. The video is the run in progress, with each page appearing when its text comes back.

One command to run it on your own pages: https://t.co/0hnnKte1GQ

[PyTorch @PyTorch]

ExecuTorch WebGPU, PyTorch models on the GPU, in the browser.

Running a PyTorch model in the browser has usually meant converting it to another framework's format first. With ExecuTorch's new WebGPU backend, you export once with torch.export, and the same .pte file runs in WebGPU browsers such as Chrome and Safari. Everything runs locally in the tab, so prompts, photos and audio never leave the device.

The backend already runs Llama 3.2, Qwen3, Gemma 3, SmolVLM, SAM 2, Depth Anything V2, D-FINE, Real-ESRGAN and Whisper, and supports LoRA and DiReFT fine-tuning on-device.

By Julian Ng-Thow-Hing (@hing3074), Siddartha Pothapragada, Stephen Jia, Digant Desai (@digant23), and Bilgin Cagatay (@BilginCatay).

Read the post: https://t.co/Cwd35vpnch
Try the models in the ExecuTorch WebGPU Playground: https://t.co/8sDhPAgrvK

[Daniel van Strien @vanstriendaniel]

See also https://t.co/uw6iwp2PGY

[Eric Bezzam @EricBezzam]

Lots of tiny TTS releases lately 🗣️

> Paradee v1.3 by @sahilmdkr
> ZONOS3-Mini by @ZyphraAI
> PiVo by @EmperoAI
> phonon by @GradiumAI
> Underdog-Woof by @UnderdogAI

Compare and listen to them side by side on the @huggingface Open TTS Leaderboard 🎧
https://t.co/R0ACuJ67B9

[Daniel van Strien @vanstriendaniel]

Recent text-to-speech models small enough to run on a CPU:

phonon-31m-en - 31M
Paradee-8M - 7.9M
Leokani-tts - 7M (4.5M version too)
PiVo - 1.4M (vocoder included)

Paradee runs ~62x real time on a single CPU thread.

[Lysandre @LysandreJik]

A pretty important new primitive just landed in 🤗 Transformers: **decision models**.

I think this could matter much more than adding support for any single model.

Today, different decision models all approach "making a decision" differently:

- @typesafeai’s Jev, the OG: typed decisions + calibrated probabilities, single forward
- @AutoTrustAI’s JEV builds open System One models around a learned decision block, w/ multimodal.
- @jaredpalmer’s KEV uses a pointer head over the options, sharing the computation of the state across questions.
- @Cloudflare’s CLEF uses a joint schema head that routes evidence between the state, questions and options, scoring the whole schema together.
- @perplexity_ai’s Decider uses a separate 255-way readout on top of Qwen, with non-causal attention in its full-attention layers.
- @liquidai’s d1 is exploring small, multimodal decision models that can run all the way down to the edge.
- And @XunzhuoLiu's Semantic Router team is pushing the serving side hard with System One + the D3 family: one typed interface for choice, score and yes/no decisions, with models from sub-1B to 27B.

That experimentation is good, and needed! But without a common interface, every training framework, inference engine and evaluation tool has to understand every implementation separately.

So Transformers is introducing a shared decision-model layer:

`DecisionConfig`
`pipeline("decision")`
and, most importantly:

`model.decide(...)`

The idea is **not** to make every decision model work the same internally, but for them to speak the same language to the ecosystem around them.

This means that, without running custom or remote code:
- inference engines can serve them through one interface
- training tools can build against common primitives.
- the Hub can package them consistently.
- evaluations can compare them without model-specific glue everywhere.

This is the part of transformers I'm the most excited about: to grow alongside the ecosystem and the many ideas; and to make all of them more usable by everyone.

https://t.co/yslV5Pc8NM

[Daniel van Strien @vanstriendaniel]

@JesseWursten @LightOnIO @TIIuae not on this one. Also find it a very good model but licence is a bit tricky so I tend to spend less time evaluating models where I also feel like there is a legal qs about when/how model can be used.

[Daniel van Strien @vanstriendaniel]

What does a private OCR eval on 20th-century documents find?

My agent and I ran 9 open-weight OCR models on 338 scanned government pages. A VLM judge compared the transcriptions in pairs, 10,730 times.

→ The top two are small: @LightOnIO OCR-3-0.8B and @TIIuae Falcon-OCR (270M)
→ The two 3B-class models came 4th and 7th
→ How you run a model matters: Unlimited-OCR-2 gained about 130 rating points with tiling

Built with ocr-bench on @huggingface Jobs. The pages stay private; only the scores are public: https://t.co/VNzQUIv5EE

v0 caveat: the judge is checked against ground truth on 19th-century books, not yet on pages like these.

IMO, more people should build private OCR evals!

Guide: https://t.co/m8N5QQrIZg

[Daniel van Strien @vanstriendaniel]

@maxmilneaus @LightOnIO @TIIuae Will maybe try at some point but already finding their small model to be very strong

[Daniel van Strien @vanstriendaniel]

nice! "Philo" score is very nice idea. Think this is the part that you also miss if you rely too much on existing benchmarks, since you often care about different things depending on what you are doing. Or you have the brutal reality of a budget, which means you really want "the best cheap model!"

[Daniel van Strien @vanstriendaniel]

@rafa_vidal_rs @LightOnIO @TIIuae Working on that but not as convinced about best approach for that!

[Daniel van Strien @vanstriendaniel]

@SomTheBuilder @LightOnIO @TIIuae Nice!

[Daniel van Strien @vanstriendaniel]

Pages like these (examples from the same collections, not from the eval set). The eval set has typescripts, letterpress and offset print, photocopies and microfilm, pages with handwriting, and 60 blank or near-blank pages since many models still mess this up still!

@LightOnIO's LightOnOCR-3-0.8B is first on 4 of the 5 page-type boards (blank pages are the exception).

[Daniel van Strien @vanstriendaniel]

@rifeash Pretty wild how small these models are getting!

[Daniel van Strien @vanstriendaniel]

Links (all on @huggingface):

phonon-31m-en: https://t.co/OjRJBr9rSK
Paradee-8M: https://t.co/j4V1g6zG0C
Leokani-tts: https://t.co/MJN1jMioqU
PiVo: https://t.co/HmkcY0USNZ

More: https://t.co/8bvsiU82S9

[Daniel van Strien @vanstriendaniel]

@staghado @antoine_chaffin @LightOnIO @CavaillesAdrien @baptaubertin This was really only really intended as a test of this method. I still think the answer is "stare at the data/failure modes"

[Daniel van Strien @vanstriendaniel]

Does this work on other OCR models? I tried a different failure: @LightOnIO's LightOnOCR-2-1B answers blank pages with "There is no content in the provided image…"

An edit to 0.13% of its weights, no training, stops it. On the BHL historical-books board it moves from 14th to 8th of 26. Model here (pls don't use it!):
https://t.co/4w7mwW98D5

This model is now one gen behind, so if you just want a good OCR model then try the new one!

[Daniel van Strien @vanstriendaniel]

OCR models often get stuck in loops, so I asked Claude Opus if it could stop one from looping by editing its weights.

It changed 0.0085% of them and with no training, it now reads this 1881 page instead of writing 16,399 characters of dots!

Prompt was basically: "read this, find a smallish OCR model and fix its repetition doom loops. You can spend up to 5 hours and $100 on @huggingface Jobs."

It only took under 3 hours in the end and about $10 of Jobs compute.

[Daniel van Strien @vanstriendaniel]

NVIDIA's model card puts NVFP4 at ~3.6% higher error than BF16 (BF16 took 5.0 min here). saturate: https://t.co/QQYGZY0QH1

[Daniel van Strien @vanstriendaniel]

Nemotron 3.5 Transcribe from @NVIDIAAI is very fast with @vllm_project!

11.3 hours of 1971 Nixon tapes on one RTX PRO 6000 (NVFP4): under 3 minutes of transcription. Whole Job, setup included: about $0.30.

For batch jobs, I run vLLM inside the @huggingface Job and send many requests at once using my tool saturate, which tunes the concurrency as it goes. A 79.7-hour run took just under 10 minutes.

Recipe: https://t.co/Sbn8NJspei

[Daniel van Strien @vanstriendaniel]

@dannyhoek @LightOnIO @MistralAI I think all their OCR models are closed? So they could try but I can't!