Zack Li

Engineering Lead, Gen AI inference @ Qualcomm | Nexa AI co-founder & CTO (acquired by Qualcomm). Opinions are my own.

[Zack Li @zacklearner]

New GenieX release from @Qualcomm AI Hub will be live today, bringing @liquidai 's LFM3 to the Qualcomm NPU / GPU / CPU for fast, private, on-device AI. 🚀
Give it a try and let us know what you build!

[Liquid AI @liquidai]

Introducing 𝙻𝙵𝙼𝟹: our new generation of Liquid Foundation Models. With a new tri-hybrid architecture and new multimodal encoders, the LFM3 family is the ideal model suite for specialized intelligence.

> 6 model instances, across text, audio, vision, and omni modalities, generative and decision models.
> 𝙻𝙵𝙼𝟹-𝟷.𝟽𝙱: text, 256k context window.
> 𝙻𝙵𝙼𝟹-𝚅𝙻-𝟸𝙱: vision-language model with video-input support for spatial understanding
> 𝙻𝙵𝙼𝟹-𝙰𝚞𝚍𝚒𝚘-𝟸𝙱: tool calls, voice chat, reasoning in 18 languages.
> 𝙻𝙵𝙼𝟹-𝙾𝚖𝚗𝚒-𝟸𝙱: an omni model combines vision, audio, and language.
> 𝚍𝟷-𝟸𝙱 and 𝚍𝟷-𝙾𝚖𝚗𝚒-𝟸𝙱: decision models with typed outputs and calibrated probabilities in a single pass. Omni adds audio input.
> fine-tune on 1 GPU.
> a new tri-hybrid architecture
> architecture optimized for 14 processor classes across GPUs, CPUs, and NPUs.
> 𝙻𝙵𝙼𝟹-𝚅𝙻-𝟸𝙱 is a beast. It performs competitively with Gemma 4 12B (6x larger), and Gemini 3.5 Flash Lite, and GPT-5.6 Luna, with 1.9B parameters.
> 𝙻𝙵𝙼𝟹-𝟷.𝟽𝙱 is #1 on the Pareto frontier of device intelligence 𝚙𝚒𝚙𝚎𝚝𝚝𝚎.𝚕𝚒𝚚𝚞𝚒𝚍.𝚊𝚒

Every modality for specialized intelligence. Pick the one that fits your use case, customize it, and run it on-device.

1/ 🧵

[Lianmin Zheng @lm_zheng]

Another mega kernel drop!
One kernel per layer. Up to 1000 token/s on muse spark open.

The finding of instruction cache is very interesting.

[Zack Li @zacklearner]

Thrilled to have you together on stage at @agihouse_org and sharing your thoughts on edge and cloud inference @zetic_ai @zeticai_

[Yeon Kim @zetic_ai]

It was such a cool experience for me to have a panel talk at @agihouse_org , together with @zacklearner ;) Love this mansion.

[Cristiano R. Amon @cristianoamon]

SpaceX selected Qualcomm Dragonwing to accelerate @Starlink Mobile’s nationwide terrestrial 5G network. Together with @elonmusk and the @SpaceX team, we’re excited to build world-leading cellular infrastructure and a path to AI-native 6G. 🤜🤛

[Zack Li @zacklearner]

I cannot agree it more @Qualcomm @QCOMResearch @elonmusk

[Elon Musk @elonmusk]

Qualcomm’s technology is excellent

[Zack Li @zacklearner]

I recently came across a very insightful post by @jiayq , Founder & CEO of Intent Lab, former VP at NVIDIA, creator of Caffe, and co-lead of @PyTorch 1.0, sharing lessons from agent-driven LLM inference optimization.Three key takeaways:

1. MegaKernel Is Not a Silver Bullet. Aggressive kernel fusion reduces launch overhead but can increase register pressure, hurt GPU occupancy, and limit scheduling flexibility. Bigger kernels don’t necessarily mean better performance.
2. Faster Kernel ≠ Faster Token Rate. A kernel can run faster in isolation yet slow down end-to-end inference due to memory contention, cache behavior, and scheduling interactions. Over 50% of their successful optimizations required no new kernels—just better execution graphs, runtime configurations, and launch ordering.
3. Verification Is the Key to Long-Running Autonomous Optimization. Reliable numerical correctness tests, end-to-end quality evaluations, and performance benchmarks allow AI agents to continuously propose, test, and validate optimizations without human intervention.

[Yangqing Jia @jiayq]

A hand-typed analysis for optimizing LLM inference with agent fleets. There has been a lot of recent news about that - but many are kinda vague, so I think a more technical post is in order.

To explain why this may claim credibility: I drew conclusion from our own DeepSeek 4.1 Flash optimization, which reached a per-step time of 2.41 ms, and a generation speed of up to 2300 tok/s (with Dspark) and reliably around 1800 tok/s for agent tasks. Agents ran ~230 optimization proposals and 94 of them were verified to be useful.

My 2 cents from those runs below:

1. Lots of gains beyond "kernel optimization". More than 50% of the optimizations were shipped with no new kernel but configs / graphs / launch order and other strategies. Kernels have been a well beaten path but there is a lot to be gained in the other parts of the end to end system.

2. MegaKernel might be a lie. On non-trivially large models, megakernel is trying to take to big a bite. Most of our experiment showed that glueing small ops into existing kernels, or fusing multiple small kernels into a medium one, is getting the biggest gain. This may only apply to NVIDIA GPUs where the underlying driver is very optimized, though, as @vllm recently showed good results with megakerkenl on TPU.

3. KernelBench and real serving are different problems. Many kernels measured positive in isolated runs but turned out negative when combined in end-to-end. I've always wanted a good simulation like Accel-Sim, but so far simulation don't have a good accuracy when modeling complicated latency patterns with both compute and memory bounds.

4. Verification becomes *really* important if you don't want to be glued to your desk. If you can define "what is correct", like numerical tests, end-to-end quality evaluations, then agents can run for a long time. We didn't interfere with the run other than (a) adding GPU machines, (b) ensure token limit aren't hit, and (c) increasing credit card spending limit.

5. Don't worry, it'll converge - I think the community has consensus that once you set up the goal, agents will eventually converge. Convergence speed is gonna be key though, and 4(c) above is really real.

6. At low concurrency settings, low level assembly writing such as SASS edits have less effect, mainly because waiting for memory (and cache?) becomes the main bottleneck.

7. The hot paths are kinda solved, although much more is to be explored. I think the biggest gain in the next couple months will come from optimizing the wider distributed serving stack, in addition to the single serving engine itself. This would probably involve a lot of system and SRE insights: building proper telemetry, building proper A/B testing without disrupting production traffic, kinda similar to the good old recommendation systems do.

8. Agents doing end to end optimization is a necessity. In our case agents diligently went through 230 experiments, each of which might take a person a week (or at least a couple days with manual experiments + claude code). The sheer volume and speed definitely allows us to explore all possible optimizations otherwise not possible due to human bandwidth.

My last thought is a bit contradictory: so far, it seems that agents are "good engineers", in the sense that they can massively carry out known insights. In a way, they are "interpolating" what we already know, but just way faster. I haven't seen agents design awesome algorithms like PagedAttention or Winograd, yet. But who knows, extrapolating from our existing knowledge base may well be very possible in the upcoming months.

And man, I like typing on a keyboard, although it might look much worse than a well trained AI writer.

[Zack Li @zacklearner]

@_JoshuaTimothy @Qualcomm @liquidai @Snapdragon Look forward to hearing more exciting progress from @_JoshuaTimothy

[Zack Li @zacklearner]

Awesome work by @_JoshuaTimothy! 🚀
Great to see @Qualcomm GenieX powering real-world agentic AI apps with @liquidai LFM3 on @Snapdragon NPUs! GenieX also is used by academic for on-device AI benchmark now, see this paper: https://t.co/wPvhH9m8Ij

[Joshua Timothy @_JoshuaTimothy]

I'm building the coolest agentic mobile app on @Snapdragon that analyzes and sorts all of your videos and photos on device! This all runs locally on the 8 Elite Gen 5 NPU using @Qualcomm_Dev GenieX and the new @liquidai LFM3 models! Video editing just got a whole lot easier!

[Zack Li @zacklearner]

Application: https://t.co/5dGY6RQWGI

[Zack Li @zacklearner]

🚀 Excited to co-host the Personal Agent to Real-World Deployment Build Day with @Lenovo × @agihouse_org! 🎉

Two years ago, we joined Lenovo Capital, @ollama, and the @GoogleDeepMind Gemma team, including @DynamicWebPaige and @jeffboudier, at AGI House to demo Nexa SDK. That year, we also unveiled AI Buddy, co-developed with Lenovo and featured by The Verge.

One year ago, Lenovo Capital invited us to CES at the Las Vegas Sphere, where we experienced Lenovo's vision for Edge AI, Cloud AI, and Hybrid AI.

This year, we're coming full circle! From on-device inference to personal AI agents, it's time to bring AI into real-world applications.

📍 AGI House, Bay Area
🔗 https://t.co/gbzahb23sF

Calling all AI Agent & AI Infrastructure founders, researchers, and builders — come connect, collaborate, and build the future of Personal AI with us! 🚀

Special thanks to @rockyrmit (Rocky Yu), Eric Yu Hai, and our friends at Lenovo and AGI House!

[Zack Li @zacklearner]

@_JoshuaTimothy @Qualcomm @liquidai @Snapdragon How Far Are We From On-Device VLM Inference? A Measurement Study on Edge NPUs: https://t.co/wPvhH9m8Ij

[Zack Li @zacklearner]

After being mocked for falling behind in AI for over a year, Google is back at #1. I think three things made the difference:
1. Distribution and real-world feedback. Search, Gemini, Workspace, and Cloud give Google access to massive real-world usage and feedback loops that most AI labs can’t match.
2. Full-stack control. From TPUs and JAX/XLA to model training and inference, Google can optimize hardware and software together, improving efficiency and iteration speed.
3. Recursive self-improvement. Better RL, reasoning, and coding capabilities mean Google’s models can increasingly help optimize their own training infrastructure and accelerate future research.

Google’s biggest advantage isn’t just having the best model today. It’s having the infrastructure and feedback loops to keep building better ones.

[Ramin @ramin_m_h]

LFM3 is here! this is an insanely elaborate release, get yourself a coffee ☕️ spend a few hours going through the blog where we tell you everything about it.

The amount of useful intelligence you can pack inside 2B parameters is really insane!

the highlights for me:

> LFM3 architecture got out of our propriety automated hardware in the loop architecture search algorithm optimized for 14 different types of processors. This is exactly the vision we embarked on: building an intelligence fabric on the fragmented device world.
> LFM3-VL-2B is truly a beast performing on par with GPT 5.6 Luna and Gemini flash ultra! at 2B parameters, wtf!!
> It is sooo easy and chip to fine-tune LFM3 instances. you can now truly build specialized intelligence for anything.
> LFM3 is enjoyably fast to train, and deploy.
> LFM3 omni 2B packs all modalities in one model enabling reliable spacial intelligence on device!
> It is now the era of products. lets build together 💫🚀

[Zack Li @zacklearner]

@_JoshuaTimothy @ramin_m_h @Snapdragon @_JoshuaTimothy wish to try it on AnythingLLM with GenieX!

[leonie @helloiamleonie]

So excited to get this out:

LFM3 is finally here!

The first release in the LFM3 family contains six models in 2B-class size:
• LFM3-1.7B: to run thousands of tiny agents in parallel
• LFM3-VL-2B: now with video support
• LFM3-Audio-2B: Speech in AND speech out
• LFM3-Omni-2B: our first ever omni model
• d1-2B and d1-Omni-2B: with new types

They are all built on the same shared LFM3-1.7B-Base.

These are super powerful to fine-tune them into tiny specialists in your domain.

Weights on Hugging Face: https://t.co/lgIW6PczfI

[Zack Li @zacklearner]

Try Liquid AI's LFM3 model with the latest GenieX v0.10.0 — just one command!

geniex infer LiquidAI/LFM3-VL-2B-GGUF

Download GenieX here: https://t.co/pmWyoUwPSx

[Zack Li @zacklearner]

New GenieX release from @Qualcomm AI Hub will be live today, bringing @liquidai 's LFM3 to the Qualcomm NPU / GPU / CPU for fast, private, on-device AI. 🚀
Give it a try and let us know what you build!

[Liquid AI @liquidai]

Introducing 𝙻𝙵𝙼𝟹: our new generation of Liquid Foundation Models. With a new tri-hybrid architecture and new multimodal encoders, the LFM3 family is the ideal model suite for specialized intelligence.

> 6 model instances, across text, audio, vision, and omni modalities, generative and decision models.
> 𝙻𝙵𝙼𝟹-𝟷.𝟽𝙱: text, 256k context window.
> 𝙻𝙵𝙼𝟹-𝚅𝙻-𝟸𝙱: vision-language model with video-input support for spatial understanding
> 𝙻𝙵𝙼𝟹-𝙰𝚞𝚍𝚒𝚘-𝟸𝙱: tool calls, voice chat, reasoning in 18 languages.
> 𝙻𝙵𝙼𝟹-𝙾𝚖𝚗𝚒-𝟸𝙱: an omni model combines vision, audio, and language.
> 𝚍𝟷-𝟸𝙱 and 𝚍𝟷-𝙾𝚖𝚗𝚒-𝟸𝙱: decision models with typed outputs and calibrated probabilities in a single pass. Omni adds audio input.
> fine-tune on 1 GPU.
> a new tri-hybrid architecture
> architecture optimized for 14 processor classes across GPUs, CPUs, and NPUs.
> 𝙻𝙵𝙼𝟹-𝚅𝙻-𝟸𝙱 is a beast. It performs competitively with Gemma 4 12B (6x larger), and Gemini 3.5 Flash Lite, and GPT-5.6 Luna, with 1.9B parameters.
> 𝙻𝙵𝙼𝟹-𝟷.𝟽𝙱 is #1 on the Pareto frontier of device intelligence 𝚙𝚒𝚙𝚎𝚝𝚝𝚎.𝚕𝚒𝚚𝚞𝚒𝚍.𝚊𝚒

Every modality for specialized intelligence. Pick the one that fits your use case, customize it, and run it on-device.

1/ 🧵

[Prince Canuma @Prince_Canuma]

Tomorrow is a huge day!

Something we've been building quietly for a while. Stay tuned, the 24h clock starts now 🚀

[Zack Li @zacklearner]

@jaschasd 's talk: https://t.co/LPZ1GTnE0X

[Zack Li @zacklearner]

I recently came across a talk by @jaschasd from @AnthropicAI at @Stanford. The phrase “last days of the Anthropocene” immediately caught my eye, and this particular slide is what really stayed with me.

As AI capabilities advance exponentially, two years of dedicated research could be overtaken by a newer foundation model before the project is even finished.

It raises a profound question: It’s no longer just “Can I solve this problem?” but “Will it still matter by the time I solve it?”

My biggest takeaway: Time is precious. Live every day fully, pursue your biggest ambitions, and focus on the hardest problems that truly matter—especially those whose value grows as AI becomes more capable.