Lequn Chen

Faster and cheaper LLM inference.

[Lequn Chen @abcdabcd987]

RT @ruihanglai: Two moments every ML researcher knows. You get onto a new cluster, and week one goes to fitting the framework to your setup…

[Lequn Chen @abcdabcd987]

We are obsessed with performance and low level details. Great work by my teammate @xyzw_io . Chat with us if this interests you.

[Perplexity @perplexity_ai]

We're open-sourcing the Unigram tokenizer we rebuilt to reduce CPU utilization by 5-6x.

Small rerankers and embedders run in single-digit milliseconds on GPU, making CPU tokenization a meaningful share of total latency.

https://t.co/QUnHeiho56

[Lequn Chen @abcdabcd987]

💻 Code: https://t.co/cOVRSbFBZD

[Lequn Chen @abcdabcd987]

🎉 Presenting at #MLSys2026 today!

fabric-lib: RDMA Point-to-Point Communication for LLM Systems

Talk by Nandor Licker at 3:15 PM. Poster 28 this evening — come say hi! 👋

[Lequn Chen @abcdabcd987]

We showcase three production applications:
🚀 KV-cache transfer for disaggregated inference.
⚡ RL weight transfer in 1.3 seconds for 1T-parameter models.
🔥 MoE dispatch/combine kernels — faster than DeepEP on ConnectX-7, fastest on EFA.

📄 Paper: https://t.co/Wech9mIqTr

[NVIDIA AI @NVIDIAAI]

Perplexity runs on NVIDIA.

Nice breakdown from the team on how they’re using the CUTLASS Python stack to optimize their models for inference 👇

[Lequn Chen @abcdabcd987]

If you are attending MLSys'26 in two weeks at Bellevue, come chat with us at our happy hour event on May 19: https://t.co/mXf7laYt1s

[Perplexity @perplexity_ai]

We’ve developed our own inference engine Runtime-Optimized Serving Engine (ROSE) to serve models ranging from embeddings to trillion-parameter LLMs.

With CuTeDSL integrated into our inference engine, Perplexity can build the specialized GPU kernels faster to bring models up to peak performance on NVIDIA Hopper and Blackwell GPUs.

[Lequn Chen @abcdabcd987]

@perplexity_ai Also check out our research paper accepted in #MLSys2026 fabric-lib: RDMA Point-to-Point Communication for LLM Systems https://t.co/Wech9mIqTr

[Lequn Chen @abcdabcd987]

#MLSys2026 is happening in two weeks! Our AI Infra team at @perplexity_ai is throwing a happy hour event at Bellevue on May 19. Come chat with us about inference, post-training, RL, kernels, GPUs, RDMA, agents, anything... https://t.co/mXf7laYt1s

[Lequn Chen @abcdabcd987]

Two years at @perlexity_ai

[Lequn Chen @abcdabcd987]

@tskaerobot @Yuchenj_UW Upload all tax documents. Prompt "prepare my 2025 tax" and your information (like location, single or married, ...). Same as what you would send to CPA. (If you don't know which docs are needed, just ask it)

[Lequn Chen @abcdabcd987]

@iamup @AravSrinivas I uploaded all tax documents and also equity contracts. Same as what I sent to my CPA previously.

[Lequn Chen @abcdabcd987]

Check out my talk at Ray Summit 2025 on RDMA Point-to-Point Communication for LLM Systems

https://t.co/sUGxsahnjD

[Lequn Chen @abcdabcd987]

zhihu: https://t.co/jH1Jo55KIG

[Lequn Chen @abcdabcd987]

Wrote a blog post on why collective communication feels awkward for newer LLM workloads (disaggregated inference, RL weight update, MoE), why people don’t just use raw RDMA, how we approached it, and some behind-the-scenes stories. https://t.co/G0IiHo54qc

[Lequn Chen @abcdabcd987]

Faster than DeepEP for Decode on ConnectX-7. First viable kernel on EFA. SM-Free RDMA transfer. Support prefill. (Maybe portable to other hardware as well)

[Perplexity @perplexity_ai]

Perplexity is the first to develop custom Mixture-of-Experts (MoE) kernels that make trillion-parameter models available with cloud platform portability.

Our team has published this work on arXiv as Perplexity's first research paper. Read more:
https://t.co/SNdgWTeO8F

[Lequn Chen @abcdabcd987]

Read more in the blog post! https://t.co/pt7w2wbV6x

[Lequn Chen @abcdabcd987]

We recently achieved 1.3-second cross-machine parameter update for Kimi-K2 (1T parameters), as opposed to a few minutes in popular frameworks.

[Lequn Chen @abcdabcd987]

We divide the weight transfer process into pipeline stages to enable overlapped execution over different hardware resources (CPU->GPU memcpy, GPU computation, RDMA, Ethernet).