Jeff Smith

AI chip monkey, former HFT automaton

[Jeff Smith @JeffSmith888]

@firstadopter No offense was taken whatsoever, and I was honestly hoping a bit to engage in a spirited debate with a real dyed in the wool skeptic. Looking forward to connecting again sometime!

[Jeff Smith @JeffSmith888]

Something obvious feeling to me now (but not long ago wasn’t) is the importance of building entire product stacks instead of just tackling the perceived core parts and assuming others will fill the gaps.

Cleverness and hard work in just a few dimensions is nowhere near enough!

[Etched @Etched]

Vertically integrating the full inference hardware stack is extremely hard.

Hear @UbertiGavin and @robertwachen share three years of battle stories leading into today's launch

[Jeff Smith @JeffSmith888]

These systems will be the first to make people reconsider their beliefs on the adequacy of GPUs for AI inference. Larger TCs, DSM tricks, etc. won’t be enough to match the throughput and efficiency of purpose-built chips.

[Etched @Etched]

We're coming out of stealth.

We've built our first racks after a successful A0 tapeout, $1B+ in customer contracts, and $800m raised.

Early customer tests show us achieving SOTA throughput, latency, and power efficiency on inference workloads.

Our first racks ship this summer.

[Jeff Smith @JeffSmith888]

@pshufb @molqtemlukni @SuchirKavi I can’t grasp the win here. Intermediate calcs still need an operand namespace (if not arch reg names), and OOO retire impls can update alias tables instead of value copies.

[Jeff Smith @JeffSmith888]

@leonard_coder @lemire What is "huge"? Simple AES-CTR gets you literal zettabits of values before reaching the "hey maybe change keys" limits for cryptographic security. I think the debate's mostly settled that MT's state space size is beyond any true practical use's fundamental needs.

[Jeff Smith @JeffSmith888]

@wmf @lemire I've heard (and personally claimed) that n*log(n) sounded a lot more reasonable, but never sub-linear!😆
Also pretty sure that Metcalfe himself admitted it was alway just a flashy marketing claim.

[Jeff Smith @JeffSmith888]

@tim_cook Sorry Tim, but whatever leadership proposed, lead, or approved of this design needs to be removed. Has Apple forgotten that mobile GUIs fundamentally require usability in adverse lighting? Forget aesthetics and computational waste, Liquid Glass will just suck in real-life usage.

[Jeff Smith @JeffSmith888]

@geofflangdale Is there a more rigorous alternative to “SIMD Instructions Considered Harmful”? The more prominently shared R-V/Berkeley/Patterson arguments have always seemed dangerously specious to me, but I’d rather not write off varlen due to wide audience arguments alone.

[Jeff Smith @JeffSmith888]

@AzorFrank I’ve been very pleased with my 6700 XT powering my home workstation, but the 9060’s specs fall short of being the upgrade I hoped for. No UBR20 was disappointing but not the deal-breaker that only 3 display outputs is.

[Jeff Smith @JeffSmith888]

@SebAaltonen @_memerao C (unfortunately IMO) chose declaration-follows-use (yay clockwise spiral rule 🙄), and I don't feel great about formatting that belies the actual grammar, so I stick with
int *ptr;
only for disliking it less, not for liking it more.

[Jeff Smith @JeffSmith888]

@icculus I’d be happy with just opt-in decoupling of layout and de/init ordering in C++, even if it could make even less pretty field lists. Cache line packing & init order deps can sometimes oppose each other. IOW, there’re at least 3 orthogonal needs with RAII style langs.

[Jeff Smith @JeffSmith888]

@geofflangdale @davidtgoldblatt Not only is 64b frequently overkill now, but AFAIK nobody even goes beyond 57b VA support yet, and LAM etc. extensions to scavenge the high bits are getting increasing attention.

[Jeff Smith @JeffSmith888]

@davidtgoldblatt The implications of 128+b VAs on TLB tag matching and PT walks for aren't fun. Likewise, 128+b scalar core datapaths (of little gain besides handling those addrs) would be expensive enough that reintroducing segmentation instead feels almost possible, if ever actually needed. 😂

[Jeff Smith @JeffSmith888]

@GawroskiT Anybody else think it's nuts to kneecap flow-through cooling performance to protect what looks to be a completely optional ARGB cable?

[Jeff Smith @JeffSmith888]

@wassickt I'm essentially always over-optimistic on these things, but I was really hoping the design would be F2F (i.e., SRAM TSVs for CCD power/GMI IO only) and maybe even W2W assembly. Thanks again for the hard work! I hope this helps pressure AMD to be more generous at ISSCC in Feb. 😄

[Jeff Smith @JeffSmith888]

@wassickt To clarify, is the CCD mounted BEOL down (towards the SRAM die) or up (flipped and using TSVs of its own)? And if it’s the former, any guesses about why the CCD wouldn’t have just been thinned by ~45um to skip the dummy 750um cap altogether?

[Jeff Smith @JeffSmith888]

@CDemerjian This sounds like declaring you’ll never use your boats to go back, but being explicitly positive above them not being burned.
Maybe too much a half measure?

[Jeff Smith @JeffSmith888]

@hkultala @SebAaltonen Arm (likely wisely) forbade anyone besides Apple from doing modal TSO toggles, but LDAR/STLR moves did get added.

The real issue isn't implicit/explicit fencing but perf. OOO cores able to handle floods of translated barriers are just a higher minimum bar than Arm wants.

[Jeff Smith @JeffSmith888]

@SebAaltonen @Gnattuoc Is this the product that will physically murder the UserBenchmark guy?

[Jeff Smith @JeffSmith888]

@SebAaltonen @Bad_AI_ Bro we’re down to half a lane now since dual-issue VLIW2-like fp32 became a thing.