Devashish Shankar

[PyTorch @PyTorch]

New from @Meta Engineering: FlashAttention-4 extended with MXFP8 support for @nvidia Blackwell—from forward and backward kernels to fused quantization and jagged cross-attention.

The team developed an end-to-end jagged module with fused quantization, FP8 activation and compute which is being used internally at Meta for GEM training. On the latest gen hardware, LP FA4 kernel reaches 2.85 PFLOP/s forward and 2 PFLOP/s backward, with up to a 1.30× end-to-end module speedup.

✍️ Devashish Shankar, Santosh Mohan, Jiaqi Xu, Darren Liu, Han Xu

Explore the design and open source implementation in our latest blog: https://t.co/KZcIxAQl1V

[Devashish Shankar @dev17392]

@SvenUrbanSci @PyTorch @Meta @nvidia The perf on jagged shapes is good! You can look at sparsity=0.5 results in the post, sparsity here represents jaggedness. Highly jagged shapes are very important to optimize for in Recsys models :)

[Engineering at Meta @Meta_Engineers]

Meta’s Generative Ads Recommendation Model (GEM), the engine behind ads on Instagram and Facebook, is now training at LLM scale across thousands of the latest-gen GPUs.

Because standard LLM infrastructure doesn't natively fit recommendation workloads, we co-designed custom kernels like Jagged Flash Attention (JFA), Generalized Dot-Product Attention (GDPA), BlockAttention, ultra-low mixed precision (including MXFP8 attention), and a topology-aware 5D parallelism architecture tailored to our network hierarchy.

The result: we scaled total training FLOPs 4x in 12 months while doubling end-to-end training efficiency to 20–25% MFU.

Read the full deep dive here: https://t.co/btMBljsNVS

[crunchbase @crunchbase]

Startup of the Day: @drishtilabs claims to be different in that it automatically digitizes human actions inside the factory to create “a massive new dataset.”

In May, two-year-old Drishti raised a $10 million Series A in a round led by @emergencecap 👉 https://t.co/dg3f02QyTi

[MIT CSAIL @MIT_CSAIL]

If Taylor Swift songs were written in JavaScript: https://t.co/7FQsNXxKTq #Reputation #TaylorSwift

[Nicolas Pinto @npinto]

#DeepLearning is like BRUTE FORCE. If it doesn't work, you're just not using enough.

[Devashish Shankar @dev17392]

AWS: 1 GCP: 0 https://t.co/7b0h908Jb0

[Alex Smola @smolix]

The new NVIDIA V100 GPUs (P3) instances are out. 1 TeraFlop per server. https://t.co/5e5Siy2BbA via @awscloud

[Insider Tech @TechInsider]

Two stars bigger than our sun just collided — this is what happened