I will offer one useful state of mind which, if one adopts it, makes everything much easier: "to accept reality as it is and to try not to regret the past and try to improve the situation." - Ilya
Megakernel blog is out now
We implement a full layer (MoE + Attention) megakernel, achieving 1100 toks/s for Muse Spark, 2.1x faster than baseline.
Nvidia's Instruction cache makes it really hard to write bigger kernels. We ended up tuning both instruction and data caches at all levels for extreme latency hiding!
With this, some warps of next operation can start running before previous operation has finished - redefining what GPU utilization means.
Full blog post: https://t.co/ubZbSQpkKl
I got 37% on ARC-AGI-2 Kaggle without LLMs!
100M looped transformer trained from scratch
- Likely highest score without an LLM
- Peaked at 6th place
- Beats last year's winning LLM
- Looping, added in the last week: 33% → 35% → 37%
thanks to @arcprize@fchollet@mikeknoop and @GregKamradt for the competition!
Link: https://t.co/iDBUrHhmdI
(writeup for the paper track)
man, who should i trust? a bunch of nerds who made linux run on apple silicon, reverse engineering that pesky gpu isa among a gazillion other things, all pre-ai.
or the guy who thinks having your agent rewrite shitty rust he never looked at in x86_64 is excellence?