Stephen Witt

The Thinking Machine, my book on Nvidia, goes on sale April 8.

[Stephen Witt @stephenwitt]

The Thinking Machine has won the FT Business Book of the Year award!

[Stephen Witt @stephenwitt]

@nabeelqu @MushtaqBilalPhD No question the Granta piece is slop, but it does lead to a more interesting question.... what can you detect that Pangram can not? And how?

[Stephen Witt @stephenwitt]

@ATabarrok @Pete_Monahan_JD Alex, why does the market fail here? Most insurance is competitively priced.

My take is that purchasers of $1MM+ assets become insensitive to add-ons ("What's another 4 grand?") introduced at the end of the buying process.

But perhaps there's more going on?

[Josh Glancy @joshglancy]

The editors @timesculture asked a few of us to pick our best read of the year.

I went for @stephenwitt's riveting biography of Jensen Huang, which helped me (sort of) understand chips but is also an extraordinary human study

https://t.co/R6um94ShLe

[Stephen Witt @stephenwitt]

For the New York Times, I wrote about ChatGPT and the race to build the world's best friend.

https://t.co/Q5s7JUhFB9

[Stephen Witt @stephenwitt]

@gls_energy Thanks George... hope all is well

[Stephen Witt @stephenwitt]

The Thinking Machine has won the FT Business Book of the Year award!

[Stephen Witt @stephenwitt]

@khoomeik To be fair, Nvidia's tensor cores do take a n-dimensional tensor and reduce it to many smaller two-dimensional mathematical objects for processing. That part is correct.

Where I erred is that it is incorrect to label the resulting matrix multiplication as "two-dimensional."

[Stephen Witt @stephenwitt]

@ben_golub Update: I'm wrong about this.

[Stephen Witt @stephenwitt]

@ben_golub I'm right about this. Tensors are downscaled to 2-D when the operation is executed in the computer. Read about it here:

https://t.co/NryYqbhrQZ

[Stephen Witt @stephenwitt]

@boazbaraktcs @aidanprattewart Actually, I really appreciate your time and insight. Thank you Boaz

[Stephen Witt @stephenwitt]

@boazbaraktcs @aidanprattewart Okay, I formally retract this one. This would not be a two-dimensional multiply.

[Stephen Witt @stephenwitt]

@boazbaraktcs @aidanprattewart Yes, sure, I mean it's all atoms and electrons in the end. But don't Nvidia's tensor cores operate at exactly the level of abstraction I'm discussing? Aren't they just little machines for producing infinity amounts of 4x4 matrix multiplies? Of simple, single-variable matrices?

[Stephen Witt @stephenwitt]

@khoomeik @boazbaraktcs @aidanprattewart Okay, got it.

[Stephen Witt @stephenwitt]

@boazbaraktcs @aidanprattewart Look, if you say I'm wrong—well, I believe you.

But it still seems to me that processing the operation at the tensor core level happens in 2-d? I must be missing something.

[Stephen Witt @stephenwitt]

The point I'm trying to make is that if you ask a data center to do a higher-dimensional tensor operation, it will break it apart and downscale it to 2-D at the level of the tensor core. So in this sense, most matrix multiplies *in the data center* are 2-D, regardless of what they look like to a programmer at a workstation.

Now, maybe that's too reductive! Or maybe, alternatively, it's not even reductive enough. After all, below this level of abstraction the operations are even more primitive. But it seems worth noting that Nvidia's empire is built on this specific operation, at this specific level of abstraction. That's the theme of this article.

[Stephen Witt @stephenwitt]

I don't dismiss it! It's probably the single-most important mathematical operation of our time. We are building nuclear power plants to do it.

But it's hard, cumbersome and unwieldy. The reason Nvidia has a $5 trillion market capitalization is because the complexity of the operation grows by the cube of the elements.

Now admittedly, non-commutativity is not *why* it's hard, but also: I didn't say that!

[Stephen Witt @stephenwitt]

@aidanprattewart @boazbaraktcs Again, tensor cores do everything in 2-D read about it here:

https://t.co/NryYqbhrQZ

[Stephen Witt @stephenwitt]

I guess the question is what level of abstraction do you want to focus on. I think for programmers at workstations this stuff is really elegant. I think if you have a Ph.D. in linear algebra you can see this as truly beautiful.

But that's not what this article is about! It's about what's being executed *inside* the data center. At that level, it's essentially tensor cores spamming 2-d matrix multiplies to infinity. (And yes, below that, it's all one-d. And below that, it's electrons being pushed around.)

It's like a building made of brick. The building is beautiful. The individual brick? Well... you kind of have to squint

[Stephen Witt @stephenwitt]

@ben_golub (note that the somewhat confusing 4x4x4 terminology refers to four 4x4 matrices, not one 4x4x4 matrix)

[Stephen Witt @stephenwitt]

@ben_golub Big reading comp fail here. Matrix multiplies for AI are indeed executed in two-d, regardless of the number of dimensions in the software framework.

The operation below is the cogwheel of capitalism.