Paul

ML and stuff

[Bryce, the CUDA Colonel @blelbach]

I do most of my development on @verdacloud B200 spot instances on Brev. They're a vital resource for the CUDA developer community!

[Dan Alistarh @DAlistarh]

Speedrunning GPT-2 is now routine thanks to @karpathy.
But can we speedrun GPT3-175B?
We attempted to match accuracy on a <$10K budget; while we didn't quite reach it, our first results show that quality data, engineering, and native FP4 can get close.
Details in 🧵

[Matej Sirovatka @m_sirovatka]

What’s the best model you can train in a day if someone hands you a pile of Blackwell GPUs?

You can try out yourself

On April 9 in Paris, @GPU_MODE + @verdacloud + @sestercegroup are hosting a GPU hackathon with a bunch of GPUs to run on and even more of them for the winners.

[Paul @edchangy]

@iamgrigorev https://t.co/3necoHr0iw does some of that

[Dan Alistarh @DAlistarh]

🚀 We are releasing state-of-the-art post-training quantization (PTQ) algorithms for Microscaling FP4, together with kernels:
- First study focused on MXFP4/NVFP4 PTQ for LLMs
- New Micro-Rotated (MR) format and GPTQ algorithm
- QuTLASS GPU kernels with up to 3.6x speedups.

[Paul @edchangy]

@antferdom @JackMonas @DataCrunch_io @scannell_aidan @rmwu36 @yizhao0313 @gpupoorftw @JackMonas, I'm just checking the submission deadline, the ICCV workshop page says 20.09, but is it now 27.09?

[WaveSpeedAI @wavespeed_ai]

👇Retweet this tweet, private message me your WaveSpeedAI account, and I'll give you guys $3 credits!🤩Only the first 100 people can get it

[Verda @verdacloud]

❗️ We just expanded our capacity of B200 SXM6 180GB servers – available in our Cloud Platform.

The best thing is…

With DataCrunch, you can deploy the Blackwell platform without approvals.

Just sign in and select the instance type:

https://t.co/psjrOd88cX

[Paul @edchangy]

Also pretty cool to see open source community building on top of each other!

[Paul @edchangy]

A new paper just dropped from Tri Dao(🐐)'s lab!

https://t.co/wr42QItFEw

Here is my hot take!

[Paul @edchangy]

Link to blog:

https://t.co/ypPHzMLN2V

[Paul @edchangy]

The paper also suggests Group Tied Attention (GTA), which works in the opposite direction and draws inspiration from MLA, incorporating those techniques into GQA.

[Paul @edchangy]

Well, the paper suggests a hybrid method. What about using MLA and adding groups?

[Paul @edchangy]

First of all, a confession! In the blog titled 'Multi-Head Latent Attention: Benefits in Memory and Computation', we didn't tell the whole story—the benchmarking on a single GPU. In reality, for DeepSeek V3-style models, parallelization is needed.

[Paul @edchangy]

Instead, one must make a copy of the latent component across GPUs, which feels wasteful.

[Verda @verdacloud]

🆕 Inference API for FLUX.1 Kontext [max] & [pro] are now available on DataCrunch!

We are an infrastructure partner of @bfl_ml for Kontext, a suite of generative flow matching models for text-to-image and image-to-image editing.

Learn more: https://t.co/BTgTKAhI0g