The Nvidia Tensor Core is the most important evolution of computer architecture in the last decade
We explain why / how it's evolved
Shout out to collaborators @bfspector@tri_dao@colfaxintl@charles_irl@ia_buck Neil Movva Jonah Alben
esp @simonguozirui for the cutest cover pic
Full Speaker List:
Mark Saroufim of GPU Mode @marksaroufim
Vijay Thakkar of Nvidia CUTLASS @__tensorcore__
Horace He of Thinking Machines @cHHillee
Philippe Tillet of OpenAI
Tri Dao of TogetherAI @tri_dao
SemiAnalysis is hosting an Nvidia Blackwell GPU Hackathon on Sunday March 16th. It is the ultimate playground for Blackwell PTX tech enthusiasts, offering hands-on exploration of Blackwell & PTX infrastructure while collaborating on open-source projects.
Here's the links for my 5-hour conversation on the future of AI with @dylan522p and @natolambert:
YouTube: https://t.co/euMFICGTuT
Spotify: https://t.co/VjTdb68p7z
Podcast: https://t.co/uxqXcfWUje
BG2. AI Compute Landscape ‘25-26, scaling intelligence, chip competition, memory tech, inference time reasoning & more. 👊💥@altcap@bgurley feat @dylan522p
Met with @LisaSu today for 1.5 hours as we went through everything
She acknowledged the gaps in AMD software stack
She took our specific recommendations seriously
She asked her team and us a lot of questions
Many changes are in flight already!
Excited to see improvements coming
MI300X vs H100 vs H200 Benchmark Part 1: Training
CUDA Moat Still Alive
Our 5 month journey conducting independent analysis + benchmarking
User Experience, Usability
GEMM + attention
InfiniBand vs Spectrum-X vs RoCEv2 Ethernet
SHARP
Total Cost of Ownership
https://t.co/A5utrO9Ht9
Happening now - the Thrilla on Chinchilla
Latent Space LIVE! - Best of 2024: Startups, Vision, Open Src, Reasoning... https://t.co/9Tvw54Svqk via @YouTube
1/5 - H100 SXM on-demand rental prices cut by 30% in the last two months. Today, you can rent H100 SXMs on-demand for as low as $2.99 per hour. What's going on?
4/5 - Blackwell's impact is another important factor. How many B100+B200s hit the Neocloud market? Will GB200 NVL36/72s be readily available in 2025 given liquid cooling tightness? How will Neocloud Giants set GB200 NVL72 debut rental price?
1/4 - Chevron deference has just been struck down by the US Supreme Court. What is it and how does it affect semiconductor companies? ⬇️
Under that 40-yr-old legal doctrine, US federal agencies had the power to create their own rules & regulations when a law is ambiguous. In our industry, this is particularly relevant for technology export controls – agencies were in the driver’s seat and didn’t have to worry about being challenged in Court. This is now over, and the power has been handed back to the Court system after the Supreme Court’s ruling in Loper Bridge Enterprises v Raimondo.
@r_bjerg @FredaDuan Freda's math is correct. H100s can't just run on their own - they need to be in a server with a CPU and various networking chips which all take up energy. Back end networking for GPU clusters - switches, optical transceivers etc also add power requirements.
@FredaDuan Other solutions could include seeking out stranded renewable resources - wind farms and such, repurposing inefficient old enterprise data centers > 1.6 PUE into AI DCs at 1.3 PUE and below.