Tianshu Yu

[email protected]. Do vision pretaining and architecture design for lfm3 vl. Democratize on-edge intelligence.

[Tianshu Yu @TianshuYu6]

@EfrainPlasc @jeffr3yli @WSJ @MercedesBenz @liquidai Glad you like the model. Working on better vision pre-training research for producing even better lfm model.

[Tianshu Yu @TianshuYu6]

@tugot17 @ramin_m_h @WSJ @MercedesBenz You should change my phone (on the left) to iphone duo from the first principle.

[Tianshu Yu @TianshuYu6]

@humayun_x @liquidai @rshubert98 d1-2B can do vision. It’s based on lfm3-vl-2B

[Liquid AI @liquidai]

Adding vision to a language model is more than one extra modality. With large images or long video, the vision encoder can use more compute than the language model itself, so efficient models need a different design. Our VLM team, including @rshubert98, @TianshuYu6, @sina_semnani, and @iamsamstevens, explains how they teach LFMs to see, from mid-training to the custom vision encoder built for LFM3.

[いずみ @w0rldisy0urz]

d1-2Bいいっすねぇ〜。jevは元々安いけど手元でdecisionモデル無限に回せるという安心感は計り知れない。何より小型なのが良い。ありがとうliquidai、ローカルLLMは沼のニオイがするなw

[Nick Mowen @nick_mowen]

@MiaAI_lab The LFM3-VL-2B is the best model for its size for video understanding, quite usable compared to all other models of that size for checking security camera footage for example.

[Tianshu Yu @TianshuYu6]

@KimiruHamiru Our omni model was not trained with thinking capability as our vl model. We will pushing for unification in the next onni release.

[Tianshu Yu @TianshuYu6]

No, it’s not public. I wrote it for training so it only work on AMD gpu. Won’t work on any device that you would use. My point is that if you want the true speed of the model you need to profile the inference engine you use. We are going to release a hyper optimized inference engine for lfm3 vl model on only mac book soon.

[花椰菜 @indesjyo]

實際體驗起來 LFM3-VL-2B 比 LFM2.5-VL-3B 厲害太多了吧！

[Tianshu Yu @TianshuYu6]

@mmm_tech If you run on llama.cpp, it will be much faster. A lot of inference engine, doesn’t have a good conv2d varshape kernel, which is critical part of the vision encoder of lfm3. I wrote customized fused kernel for training.

[irony.aditya @0xironyAditya]

LFM3-VL-2B vs Gemma 4 E2B on 50 real scanned receipts.

LFM3 got 41 of 50 receipts fully right. Gemma got 28. 50 receipts from SROIE, same prompt, JSON out, one 5060 Ti 16 GB.

- LFM3-VL-2B: 95% of fields right, 41/50 receipts perfect, 46 s
- Gemma 4 E2B: 87% of fields right, 28/50 receipts perfect, 62 s

Across both tests: Gemma reads messy chat-style text a bit better (95% vs 90%). LFM3 reads receipts better, and was faster both times.

@liquidai @googlegemma

[Doğukan @DogukanUrker]

pointed lfm3-vl-2b at the whole san fernando valley overnight. 16,692 aerial tiles, one rtx 3060, 5.5 hours.

it found 62,873 pools and drew a box around each one.

not a crazy idea, just “find the pools”. it misses about 1 in 5 and boxes a roof now and then. but it’s a 2B model with zero finetune.

imagine what you can do with this. palantir at home lmao

[Tianshu Yu @TianshuYu6]

@pipisha Write your own on phone serving engine or wait for the pull request for llama.cpp to be merged

[Tianshu Yu @TianshuYu6]

@DogukanUrker Crazy. How did you fit the high res map into the model? I don’t remember putting ton of these satellite data during pre-training. Crazy OOD performance.

[Tianshu Yu @TianshuYu6]

@minamotomasaru The performance gap between audio model and omni model is very small. The gap between vision model and omni, text model and omni are much bigger. We are working unifying those modality into the same model better. It’s our first attempt.

[Tianshu Yu @TianshuYu6]

@iamsamstevens @vincentzed_cuda Everything else are just unnecessary abstraction

[Tianshu Yu @TianshuYu6]

@vincentzed_cuda When you are back, we gonna get roasted baby pigeon again

[Tianshu Yu @TianshuYu6]

@btckad @liquidai Maybe or maybe not

[Tianshu Yu @TianshuYu6]

@970426Com @liquidai You can just do things

[Tianshu Yu @TianshuYu6]

@NNightmare84631 @liquidai Size doesn’t matter