Shuming Hu

distinguished machine not learning engineer

[Shuming Hu @ShumingHu]

@suchenzang grok gave me this link for more details
https://t.co/MzYALKATw0

[Shuming Hu @ShumingHu]

@vincent_jh_cho Yes. But too bad ,unlike this one, I cannot parse/verify those…

[Shuming Hu @ShumingHu]

@JeffDean @giffmana I read this from PaLM paper :)

[Shuming Hu @ShumingHu]

@deanwball Easy, remove oai affiliation and the like from your bio.

[Shuming Hu @ShumingHu]

@tony_yet @ethayarajh How about zhihu?

[Shuming Hu @ShumingHu]

@suchenzang @giffmana A pic is worth 16x16 words

[Shuming Hu @ShumingHu]

@dhruv31415 @timorchik @beyarkay nah older is better than younger!

[Shuming Hu @ShumingHu]

@aryaman2020 不客气

[Shuming Hu @ShumingHu]

@dhruv31415 Any objective is allowed? I might have a way

[Shuming Hu @ShumingHu]

@VictorTaelin DL is not built for erasure as much as we don’t control what we forget.
It’s super important part of intelligence but it’s not the same as tasks like compacting text paragraphs.

[Shuming Hu @ShumingHu]

OTOH, I don’t see any fundamental difference among different compute languages humans can read and write. If transformers can do python well, why not others 😅

[Shuming Hu @ShumingHu]

My gut feeling. Thoughts?

[Shuming Hu @ShumingHu]

Formal languages can be verified by a computer where computers are very good at state tracking.

Informal languages can be parsed and understood by huamns, who have less working memory?

Maybe informal languages are better fit for transfomers, which are not good at state tracking and may have some other limitations in latent reasoning?

[Shuming Hu @ShumingHu]

Formal languages can be verified by a computer where computers are very good at state tracking.

Informal languages can be parsed and understood by huamns, who have less working memory?

Maybe informal languages are better fit for transfomers, which are not good at state tracking and may have some other limitations in latent reasoning?

[Shuming Hu @ShumingHu]

K3 is better at optimizing AttnRes kernel, which is also used in K3, than frontier models :)

Huge congrats to Kimi team! Open weight models are catching up.

[Kimi.ai @Kimi_Moonshot]

Self-evolving: AttnRes Kernel Optimization

Given FLA Triton AttnRes at production scale (96 layers, 8192-dim model, 8192 tokens), the goal was to maximize training-side speed without changing numerics.

Over 15 hours of nonstop iteration, K3 designed a novel two-phase kernel algorithm, fused kernels while preserving numerics, and reduced forward+backward time from 283.6 ms to 114.4 ms.

K3 and Fable-5 (with potential fallback) reached similar performance, but K3 improved faster per iteration.

[Shuming Hu @ShumingHu]

The more you want to treat it like AGI, the more likely it dodges you.

[Shuming Hu @ShumingHu]

@suchenzang Susan is in her Chinese cursing phase

[Shuming Hu @ShumingHu]

Naively, good quantization results mean original model is at flat loss basin.

[Shuming Hu @ShumingHu]

What are some of the theoretical explanations for why 1-bit model (post-training quantizations) work so well?

[PrismML @PrismML]

The footprint reduction does not come at the expense of the capabilities that matter.

Across 15 benchmarks spanning knowledge, reasoning, math, coding, instruction following, tool use, and vision, Ternary Bonsai 27B retains 95% of the full-precision model’s performance. The 1-bit variant retains 90%.

Math and coding remain especially strong, while tool-calling performance stays close to full precision.

[Shuming Hu @ShumingHu]

@cosminnegruseri nah they didn't change sparsity.

I think they only have sign bit. it's probably technically 1+ bit since they store a 16 bit scaling for every 128 param group as well (1+ 16/128)

[Shuming Hu @ShumingHu]

i think sparsity and lottery theroy ticket line of thinking is more akin to
full model -> pruned model
while keeping most of the performance

Here it's full model -> lower precision -> bit model
but didn't change sparsity or network topology.
AFAIK this only works for some networks and especially networks above certain size.