Jerry Zhou

exists as 451; opinions are my own; Creator of @LANDropApp, @AthenaAGI, LMRouter; MTS @MicrosoftAI LLM training infra; Ex-@NVIDIA RISC-V security; CN @yvbbrjdr

[Jerry Zhou @realJerryZhou]

Happy to help!

[SGLang @sgl_project]

🚀 SGLang now officially supports @thinkymachines Inkling-Small running on 2× @NVIDIAAI DGX Spark systems connected via ConnectX-7.

Huge thanks to @realJerryzhou @ispobaoke for implementing the support, with valuable community contributions from @u1tra_instinct and @TechMDAI.

This enables strong agentic capabilities to run locally on a compact platform. The current setup achieves 24 tok/s (concurrency=1) without MTP.

DSpark support is coming soon.

Check out the cookbook below to get started 👇

[SGLang @sgl_project]

🚀 SGLang now officially supports @thinkymachines Inkling-Small running on 2× @NVIDIAAI DGX Spark systems connected via ConnectX-7.

Huge thanks to @realJerryzhou @ispobaoke for implementing the support, with valuable community contributions from @u1tra_instinct and @TechMDAI.

This enables strong agentic capabilities to run locally on a compact platform. The current setup achieves 24 tok/s (concurrency=1) without MTP.

DSpark support is coming soon.

Check out the cookbook below to get started 👇

[Jerry Zhou @realJerryZhou]

What makes the attention mechanism work is the fact that we perform matrix multiplication between activations, compared with activation × weights.

[Jerry Zhou @realJerryZhou]

Dobby is never free

[Jerry Zhou @realJerryZhou]

A lot of concepts in ML perf optimizations have 1-to-1 mapping to human collaboration in a team if you think about it. We as human also have "compute throughput", "memory throughput", communication overhead, straggler issue, all reduce, different kinds of parallelism, etc.

[Jerry Zhou @realJerryZhou]

mind blowing that cute dsl bf16 gemm is faster than cublas

[Mingyi Lu @mingyilu123]

Everything you need to know about how we made this magic happen!

Where the speedup comes from:

- Spec V2 overlap runtime: zero bubbles between decode iterations, +11% TPS
- IndexShare MTP: reuse indexer top k across draft steps, up to 1.9x cheaper drafts at long context
- TopK-V2: selection instead of sorting, 2.33x at 80K ISL, 10.17x at 1M, interactivity stays flat
- Indexer prologue fusion: 12 kernels to 4, +8% decode at bs=1
- CuTe DSL BF16 GEMM: beats cuBLAS at every decode batch size, +4%

[Jerry Zhou @realJerryZhou]

If we do live in a simulation, then creating intelligence - if it's even possible - might look like running Minecraft inside Minecraft itself with redstone - not very efficiently. Basically computational irreducibility.

[Jerry Zhou @realJerryZhou]

No one is going to complain that compilers took their assembly-writing jobs.

[Jerry Zhou @realJerryZhou]

LLMs
have
NO
freakin'
taste

[Jerry Zhou @realJerryZhou]

@mingyilu123 so cute

[Jerry Zhou @realJerryZhou]

A lot of critics are missing the point about MAI models - it's way easier to cheat on benchmark scores than to produce an actually useful model.

[Jerry Zhou @realJerryZhou]

Machine learning is, in fact, a digital signal processing problem.

[Jerry Zhou @realJerryZhou]

If this continues perpetually, there will be a point where you can't even repay the interest, and the entire thing defaults and collapses.

I'm not against vibe coding. Just be mindful about vibe coding for a production service. This can and will get ugly.

[Jerry Zhou @realJerryZhou]

Unpopular opinion: Vibe coding is a mega form of taking out a loan in the form of tech debt. Constant vibe coding means your balance only increases. You can't dodge repaying the loan eventually. You're just delaying the process.

[Jerry Zhou @realJerryZhou]

Polished achievements outside, constant struggle underneath.

[galaxy @galaxy1025850]

致我的学生们
有些时候，特别是在今年，我会感到绝望，甚至怀疑我究竟在让你们为什么做准备。软件行业正变得完全疯狂，更不用说政治气候了。将你们培养成计算机科学家，然后把你们送入这样一个世界，感觉几乎是不道德的：在那里，初级计算机职位很难找到；知识产权不受尊重
https://t.co/UQOtiQafrL

[Jerry Zhou @realJerryZhou]

I think of Earth as some kind of giant simulation, much like Minecraft, where people participate and build. They eventually "die" in this world, but this is the only way things can progress and I don't think death is the end for your soul, only your current physical presence.

[Jerry Zhou @realJerryZhou]

1. Why is the gap between human intelligence and other animals' intelligence so large?
2. Why can we make something that's so close to human intelligence yet far worse than other animals' intelligence?

[Jerry Zhou @realJerryZhou]

Reinforcement Learning is inherently flawed. We can be rewarded for the end goal, but the process to it is not necessarily rewarding.

[Jerry Zhou @realJerryZhou]

The act of semi-acquihiring Groq is an act of panic—just like Facebook acquiring Instagram.

(Disclaimer: the em-dash is not written by an AI)