Desmond Mehta

building @opt32co | prev @yale

[Peter0x44 @peter0x44]

x86_64 has the mandate of heaven

[tender @tenderizzation]

the optimal pytorch core maintainer workstation has two physical cpu sockets (to run both x86 and ARM binaries natively) with a minimum of 64 physical cores of each (to build ATen and Flash Attention), one SXM6 socket (for SM 10.0/3), 8 PCI-E slots (for Titan Xp, Titan V, 2080 Ti, Ampere, GeForce Ampere, Small Ampere like A10/A16/A2, Ada, and RTX Blackwell), a separate Grace Hopper or Grace Blackwell board for UVM-specific work, a Jetson Thor sitting on top (for SM 10.1), all fully passively cooled (to facilitate coil-whine debugging), with support for a diverse set of networking configurations

who's building this?

[Desmond Mehta @Dezash3]

@maharshii Fused-quant gemms are still a pretty idealized case

[Desmond Mehta @Dezash3]

@LottoLabs They are not good

[LBO_Guy @The_Analyst_Lab]

Why is everyone comparing EMIB to CoWoS-S?😭🤣
CoWoS-S is the H100 generation. TSMC moved to CoWoS-L (organic RDL + embedded silicon bridges) for Blackwell, which is already shipping AT SCALE. This already scales beyond reticle limits. The actual roadmap though is CoPoS (the same RDL+bridge interposer on a glass panel), which attacks the size and cost limits more fundamentally than EMIB does. EMIB is lagging so badly atm, lol.

[high cagr @HiCagr]

CXL & flash

[Desmond Mehta @Dezash3]

Comically unoptimized

[pabs @pabloberlangab]

"run the model on the robot: cloud is too slow."
been experimenting. Same VLA, same arms, same task.

our custom cloud engine: 5.1x faster compute. 1.7x faster end-to-end round-trip
(more in thread)

[SemiAnalysis @SemiAnalysis_]

100% of AI chip startups have slides/“simulated performance data” showing that their chip is way better, but 99% of custom ASICs fail. Why? The MATH isn’t MATH until you realize that AI chips are about Software. It is relatively easy to build a chip and put numbers onto slides; it is hard to build great software. That is why 99% of AI chip startups fail.

[Desmond Mehta @Dezash3]

@HighPriestOfSWO The world isnt ready for fruit box

[Underfox @Underfox3]

@IanCutress If we're going to do a "What if" exercise, we should remember that Intel had the technology to transform optante into an in-memory tensor accelerator in 2019.

Now, imagine that combined with Gaudi 3.

https://t.co/X2hmjFy2kE

[i2cjak @i2cjak]

everyone trying to do cursor for electronics primarily writes software and does electronics as a secondary thing. Nope, has to be a true electronicscel who so happens to write software to succeed. I think this is where Diode probably has an edge

[Ryan Yang-Liu @ryanyang0]

I'm an nvidia's fanboy but it's kindof nuts that they sold the DGX Spark as a "1 Petaflop Minicomputer*"

with the asterisk being that it's sparse fp4. ok sure.

so they get to 1P by taking 500T for dense fp4 and 2x-ing it but didn't bother checking if you can actually hit this #?? and still no tcgen05 seven months later

[Desmond Mehta @Dezash3]

@JustJake Maybe not for long…

[Desmond Mehta @Dezash3]

@jino_rohit Yes exactly, ampere tensor cores do support int8 though just not fp8. Generally looks like doubled ops per cycle when halving precision (and then nvidia often advertises double again because they factor in 50% sparsity)

[Ryan Yang-Liu @ryanyang0]

it's wild how if you do a Exact Match search for any seminiche PTX instruction, your only two results will be official CUDA docs and HazyResearch

[Daniel Kuntz @dankuntz]

The perfect silicon for your thing already exists. It’s just not in stock and you have to do whippets with a guy in Hangzhou to maaaaaybe get a small slice of allocation in 10 months

[Desmond Mehta @Dezash3]

@PatrickToulme @bendee983 LLMs in production are not deterministic

[Desmond Mehta @Dezash3]

@Spotify please raise ur algo temperature

[Joshua Kushner @JoshuaKushner]

make data centers aesthetically beautiful