Naman

🇺🇸🇳🇿🇮🇳 Interests: GPUs, AI compute and Inference, History. Indophile. Tweets reflect my own views.

[Naman @IamNamanM]

@BenBajarin 💯

[Naman @IamNamanM]

@ai Thank you Anand.

The OCP announcements really bring the speed of adoption of disaggregated inference to light…which for perspective had almost non existent mindshare before the Groq -NVIDIA announcement.

[Naman @IamNamanM]

@tsait26 Great post. Thank you

The KV cache transfer is the part most interesting for me. More to learn in this space everyday

[Naman @IamNamanM]

@JayaGup10 Great read of the market. Thank you

[Daniel Newman @danielnewmanUV]

Bro. 😎😂

[Naman @IamNamanM]

@pitdesi @DLoesch Air NZ

[Naman @IamNamanM]

@jmrphy Amen! Dwarkesh is impressive in his knowledge, candor and skills as a podcastor/interviewer.

People just can't tolerate someone doing something that isn't exactly how they prefer it. That's the crux of all the ridiculing and hating.

[Naman @IamNamanM]

@parrysingh 😂😂

[Naman @IamNamanM]

@pitdesi Yeah the cleaning experience took away some of the benefits of not chaperoning

[stepfanie tyler @stepfanie]

the penalty for destruction is zero

the penalty for production is 37%

[Naman @IamNamanM]

Great video on the importance of positive relationships!

What actually makes a good life, according to 85 years of data | Robert ... https://t.co/EMftjXweRa via @YouTube

[Gavin Baker @GavinSBaker]

Nvidia is buying Groq for two reasons imo.

1) Inference is disaggregating into prefill and decode. SRAM architectures have unique advantages in decode for workloads where performance is primarily a function of memory bandwidth. Rubin CPX, Rubin and the putative “Rubin SRAM” variant derived from Groq should give Nvidia the ability to mix and match chips to create the optimal balance of performance vs. cost for each workload. Rubin CPX is optimized for massive context windows during prefill as a result of super high memory capacity with its relatively low bandwidth GDDR DRAM. Rubin is the workhorse for training and high density, batched inference workloads with its HBM DRAM striking a balance between memory bandwidth and capacity. The Groq-derived "Rubin SRAM" is optimized for ultra-low latency agentic reasoning inference workloads as a result of SRAM’s extremely high memory bandwidth at the cost of lower memory capacity. In the latter case, either CPX or the normal Rubin will likely be used for prefill.

2) It has been clear for a long time that SRAM architectures can hit token per second metrics much higher than GPUs, TPUs or any ASIC that we have yet seen. Extremely low latency per individual user at the expense of throughput per dollar. It was less clear 18 months ago whether end users were willing to pay for this speed (SRAM more expensive per token due to much smaller batch sizes). It is now abundantly clear from Cerebras and Groq’s recent results that users are willing to pay for speed.

Increases my confidence that all ASICs except TPU, AI5 and Trainium will eventually be canceled. Good luck competing with the 3 Rubin variants and multiple associated networking chips. Although it does sound like OpenAI’s ASIC will be surprisingly good (much better than the Meta and Microsoft ASICs).

Let’s see what AMD does. Intel already moving in this direction (they have a prefill optimized SKU and purchased SambaNova, which was the weakest SRAM competitor). Kinda funny that Meta bought Rivos.

And Cerebras, where I am biased, is now in a very interesting and highly strategic position as the last (per public knowledge) independent SRAM player that was ahead of Groq on all public benchmarks. Groq’s “many chip” rack architecture, however, was much easier to integrate with Nvidia’s networking stack and perhaps even within a single rack while Cerebras’s WSE almost has to be an independent rack.

[Naman @IamNamanM]

@danielnewmanUV Datacenters demand is not going to decelerate in the near future

[Naman @IamNamanM]

@RihardJarc Google’s TPU moat is underestimated. Once they figure out the business model to sell it general purpose, NVIDIA would have a real challenger for market share.

[Naman @IamNamanM]

@RihardJarc Hence the need for older air cooled GPUs to stay within enterprise data centers or colos catering to enterprises. LC infra is not there yet, which means your typical AC L40S or H200 NVL will have demand till NVIDIA provides support for them.

My 2c. loved the analysis as usual

[Rohan Paul @rohanpaul_ai]

AI chip roadmap for Nvidia, AMD, Intel, Google and others.

[Naman @IamNamanM]

👏🏼👏🏼

[David Sacks @davidsacks47]

Congrats to Joe Gebbia on becoming the first Chief Design Officer of the United States. In this newly created role, Joe will oversee the redesign of roughly 26,000 federal web portals, many of which are obsolete, so they better serve social security recipients, veterans, and all citizens.

As a former cofounder of @Airbnb, @jgebbia could have done anything next but decided to volunteer his time and expertise to public service. Thank you Joe.

And thank you to President Trump for creating this new function by executive order and continuing to attract the best talent from the private sector to join his administration. It’s an honor to be part of!

[Packy McCormick @packyM]

One of America's most important companies, Vulcan Elements, is being built in one of its greatest cities (Durham, NC).