Bram Wasti

working on fast inference things with msl

[Bram Wasti @bwasti]

@cHHillee 4. make more layers share global attn - yoco/shared kv

[Jack Zhang @jcz42]

We made Muon run up to 2x faster for free!

Introducing Gram Newton-Schulz: a mathematically equivalent but computationally faster Newton-Schulz algorithm for polar decomposition.

Gram Newton-Schulz rewrites Newton-Schulz such that instead of iterating on the expensive rectangular X matrix, we iterate on the small, square, symmetric XX^T Gram matrix to reduce FLOPs. This allows us to make more use of fast symmetric GEMM kernels on Hopper and Blackwell, halving the FLOPs of each of those GEMMs.

Gram Newton-Schulz is a drop-in replacement of Newton-Schulz for your Muon use case: we see validation perplexity preserved within 0.01, and share our (long!) journey stabilizing this algorithm and ensuring that training quality is preserved above all else.

This was a super fun project with @noahamsel, @berlinchen, and @tri_dao that spanned theory, numerical analysis, and ML systems! Blog and codebase linked below 🧵

[Arthur Spirling @arthur_spirling]

🚨BREAKING AI NEWS 🚨

A Cambridge study just dropped that PROVES you can exactly calculate the slopes of functions at an arbitrary point. This UNLOCKS gradient optimization that experts say is vital for AGI.

Download our app for a daily AI digest delivered to your inbox

[Bram Wasti @bwasti]

gemini gets it tho. 👏

[Bram Wasti @bwasti]

pretty much no AI can classify a hand painted CIFAR-100 clock. a very under-explored dataset!

[Bram Wasti @bwasti]

using vim to edit my prompt

[Bram Wasti @bwasti]

when claude comes up with a root cause it already debunked three chat compressions ago

[Bram Wasti @bwasti]

yep, definitely a big part of the challenge and the reason I'm starting with a fixed size font for now. I'm moving from pure frame prediction to delta prediction (over char-by-char rendnering) which will only require learning next position/shape of the letters on top of the actual latent text structure

big issue has been finding a good loss to capture the low freq position data and the high freq font-shape data at once

[Bram Wasti @bwasti]

new side project, stay tuned :)

[Kimi.ai @Kimi_Moonshot]

🥝 Meet Kimi K2.5, Open-Source Visual Agentic Intelligence.

🔹 Global SOTA on Agentic Benchmarks: HLE full set (50.2%), BrowseComp (74.9%)
🔹 Open-source SOTA on Vision and Coding: MMMU Pro (78.5%), VideoMMMU (86.6%), SWE-bench Verified (76.8%)
🔹 Code with Taste: turn chats, images & videos into aesthetic websites with expressive motion.
🔹 Agent Swarm (Beta): self-directed agents working in parallel, at scale. Up to 100 sub-agents, 1,500 tool calls, 4.5× faster compared with single-agent setup.
-
🥝 K2.5 is now live on https://t.co/YutVbwktG0 in chat mode and agent mode.
🥝 K2.5 Agent Swarm in beta for high-tier users.
🥝 For production-grade coding, you can pair K2.5 with Kimi Code: https://t.co/A5WQozJF3s
-
🔗 API: https://t.co/EOZkbOwCN4
🔗 Tech blog: https://t.co/6h2KkoA0xd
🔗 Weights & code: https://t.co/H38KegeDIY

[Bram Wasti @bwasti]

@rustyryan still playing with the exact setup, but directly over pixels means an output dim of 16k which isn’t terrible at first glance…

[Bram Wasti @bwasti]

ive found a key to complex systems dev with AI involves iterating heavily on a logging system as a first class citizen. that lets the AI reconcile complex logic against what actually happens (races, slowdowns, deadlocks etc)

stream based designs lend themselves well to that (https://t.co/jUVsh1KfwN)

[Bram Wasti @bwasti]

contributing to a short talk at the vllm office hours! https://t.co/cAAakJAqeU

[Bram Wasti @bwasti]

would be so nice to have claude venvs. `source my_claude_env/activate` without having to check your dir for the configs/mds

cc @bcherny

[tender @tenderizzation]

prefill with a single token be like

[Bram Wasti @bwasti]

@drisspg @typedfemale how does it compare to nick + sons?

[Anand Gopalakrishnan @agopal42]

Our new paper shows that RoPE—the positional encoding used in most modern LLMs like Qwen, Gemma, DeepSeek—has a fundamental flaw: it entangles "what" (content) and "where" (position) information.
Our fix (PoPE) is simple but powerful. Paper: https://t.co/XlltfcSwHQ

[Bram Wasti @bwasti]

@CSProfKGD I remember implementing HOG features in SIMD (SSE2!) and then again in pure javascript (before web assembly)

[Bram Wasti @bwasti]

do we really need things like garbage collection in an AI-native world? how much of the overhead in high-level languages can just disappear with vibe-coded stacks?

zig is looking more appealing every day

[Bram Wasti @bwasti]

github:bwasti/binfer -- an experiment with fast inference serving using bun + cuda.

trying to focus on speed / UX experiments, like language overhead and startup time. not trying to maintain this as a legit framework (use vllm or sglang)