Just entered the @paradigm Open Kernel Challenge: make Meta's Muse Spark Open model run faster on H200 GPUs.
First submission: 1.10Γ faster than SGLang. Complete beginner, learning as I go.
https://t.co/P1KdfCQjVy
@nixxin > Someone said to me that India is consuming 20% of the worlds compute. We should use Chinese models and serve them locally to hopefully reduce the foreign exchange outflow.
These are home grown GPU ? we are still spending on buying GPU ?
Indian government still not aware about difference using GUPs for hosting vs training/fine tuning.
They will probably subsidize more GPU for Sarvam, tax money spent well.
Today, we are launching Sarvam Token Factory for everyone.
We host our own models, and the full range of best-in-class open-weight models, in India. We match leading inference providers on performance and reliability.
For enterprises, this means more models to choose from, and an AI stack they can own without lock-ins.
@paradigm I week I didn't know what a GPU kernel was. Today my Triton kernels run Meta's Muse Spark router 4.6Γ faster than SGLang on an H200.
The biggest lesson: most of the time goes to moving memory, not maths.
#935 of 1634 on projections.
Just entered the @paradigm Open Kernel Challenge: make Meta's Muse Spark Open model run faster on H200 GPUs.
First submission: 1.10Γ faster than SGLang. Complete beginner, learning as I go.
https://t.co/P1KdfCQjVy
Today, I was having discussion with my director in office related to skills he is looking out in next hire.
He said : combination of sales + engineer + product thinking, may be some other combo of skills.
It's time to expand our expertise horizon.
@paradigm And because x Γ (hi + lo) = xΓhi + xΓlo, two fast short multiplies give nearly the same answer as one slow long one
Result : Router step: 26 Β΅s ==> 9 Β΅s
@paradigm The model's "router" picks which experts handle each word, so it must be accurate. That's why it uses slow FP32.
The trick: split each weight into two short numbers.
real: 0.0123456791
hi: 0.01233
lo: 0.0000157 (the leftover)
hi + lo = 0.0123457 almost exact