Hui United

I am a songwriter. In my previous life, I worked as a software engineer.

[Hui United @HuiUnited]

@MrMessenger Amazon CEO ก็อธิบายเหตุผลการลงทุนอย่างดีนะครับ หุ้นถึงขึ้นหลังจากประกาศผลเมื่อสองสัปดาห์ก่อน

[Hui United @HuiUnited]

@firstadopter If you want a team with history and huge fan base worldwide, check out Liverpool.
https://t.co/soZqAVWfjo

[The Players’ Tribune @PlayersTribune]

“I wrote this because I can’t speak about it.

I wrote this because I want you to know that I will make sure that you live on.”

@RBLeipzig and @equipenatciv winger Yan Diomande on the life of his sister, Roxane. https://t.co/6wQmpdWTSi

[Amy Deng @amydeng_]

I’m an AI researcher turned brain tumor patient, and recently I used the models to crack my mystery fatigue faster than my PCP could.
I believe everyone can do the same with their own symptoms. Here’s how:

[Mayank Vora @aiwithmayank]

Steven Pinker is a Harvard psychologist who wrote the only style guide based on how the brain actually reads.

Here are 10 writing fixes from "The Sense of Style" rooted in cognitive science, not grammar rules.

1) The curse of knowledge ruins more writing than laziness

[Hui United @HuiUnited]

@jasonstandup @MrMessenger เห็นด้วยเลยครับ ให้รัฐมาถือหุ้นด้วยน่าจะช่วยลดความเสี่ยงลงไปได้เยอะ

[David Yarrow @David_Yarrow]

The Vikings Are Coming 🇳🇴⚔️

in collaboration with the Norway FF

[Hui United @HuiUnited]

@MrMessenger เรื่องความสัมพันธ์หลังฉากเราไม่มีทางรู้ เรื่องสไตล์การเล่นถกเถียงกันได้ว่าดีไม่ดีอย่างไร (ย้อนแย้งมากที่ซาล่าห์อยากเล่นheavy metal football แต่เค้านี่แหละที่เล่นแบบนั้นไม่ได้แล้ว) แต่เรื่องซาล่าห์ผมว่าสล็อตทำได้ดีครับ

[Hui United @HuiUnited]

@MrMessenger ขอออกความเห็นในฐานะแฟนหงส์คนนึงนะครับ ผมมองว่าการดร็อปซาล่าห์ในช่วงนั้นเป็นสิ่งที่ถูกต้องนะครับ ซาล่าห์ฟอร์มตกอย่างรุนแรง เลี้ยงผ่านใครไม่ได้เลย ไม่ใช่คนเดิมในตำนานอีกต่อไป responseของสล็อตต่อสิ่งที่ซาล่าห์ทำทั้งที่ลีดส์และเมื่อสองวีกก่อนก็ถือว่าhandleได้ดีหลีกเลี่ยงการเผชิญหน้า

[fin @fi56622380]

AI Semiconductor Endgame 2026 (Part 1)

New Token Economics Computing Paradigm Shifts from GPU Compute to HBM

This article starts from the essence of GPU architectural evolution to address a question the market has long worried about:

Why must each GPU's HBM memory demand grow exponentially, and why won't this exponential growth in HBM demand stall?

It then derives the first principle of token economics under the current architecture: token throughput = HBM size × HBM BW (bandwidth)

It also discusses why the GPU ceiling is determined by HBM's two dimensions of progress.

The topic of HBM cyclicality has long been controversial. Optimists argue that AI-driven demand is much greater than before, but the market mainstream still believes that previous up-cycles also saw 20%+ annual demand growth — so what's different this time? AI doesn't change the fact that HBM, like traditional DRAM, has commodity attributes. Once capacity expansion at the demand peak meets a downturn, history will repeat itself. We can take the perspective of compute-chip architecture, start from first principles, and unpack and reason through this question:

why this time is genuinely different.

———————————————————————————————

History: The Era of CPU Compute

For a very long time, we lived in the era of CPU-dominated compute. The CPU's top-level KPI was performance — running faster — and so each generation of CPUs deployed every method imaginable to push benchmark scores higher. First it was rising clock frequencies, then it was architectural evolution: superscalar designs, and so on.

During this period, why didn't DDR need to advance technologically at high speed? DDR3 to DDR5 took a full 15 years.

Because in this era, DDR's role was purely auxiliary — and only weakly so. By industry experience, even doubling DDR speed would generally only raise CPU performance by less than 20%.

Why did improvements in DDR bandwidth and speed matter so little? Two reasons:

1. CPUs designed all kinds of architectural tricks to hide DDR latency — superscalar designs, wider issue widths, massive ROBs and register renaming to extract parallelism and hide latency, L1 caches, L2 caches — all of which weakened the demand for DDR bandwidth and speed.

2. CPU workloads don't have particularly demanding bandwidth requirements. For most everyday workloads — say, opening a webpage — DDR bandwidth is severely overprovisioned. Even cloud workloads often look the same.

In other words, in the CPU era, DDR bandwidth and speed didn't really matter. There was virtually no difference between DDR4 and DDR5 except in a handful of games — and even the JEDEC standard advanced slowly.

On top of that, only a small portion of any given app needs to permanently sit in DDR. Whatever is needed can be paged in from the hard drive on demand. App size grew slowly, and so DDR capacity demand grew slowly as well.

That's why, over the past decade, the average PC went from 7–8GB of DDR to about 23GB — only 3× growth in ten years.

This slow upgrade pace directly affected revenue. Capacity-based pricing was the main way of making money; speed improvements were just a technological upgrade that raised the unit price of capacity. With both of these dimensions advancing slowly, growth could only come from increases in PC/phone unit volumes.

So along both dimensions — bandwidth/speed and capacity — DRAM was always a “nice-to-have” appendage to the chip industry. The marginal utility of DDR upgrades was very low, and almost completely disconnected from the CPU era's top-level KPI.

———————————————————————————————

The Paradigm Shift: GenAI's Top-Level KPI

When we entered the era of GenAI large models, the computing paradigm shifted, and the top-level KPI changed fundamentally.

By the time GPUs evolved into AI inference engines, the top-level KPI was no longer compute alone (TOPS/FLOPS), as it had been for CPUs — it became the cost of a token. Specifically: overall token throughput per unit cost / per unit power.

A close second is token throughput speed — because in the agent era, many tasks have become serial, and token output speed has become a critical bottleneck for user experience.

This is exactly why Jensen invented the concept of the AI factory: to produce the most tokens at the lowest cost, while pushing token throughput speed as high as possible.

In the AI training era, Jensen's economics were TCO (Total Cost of Ownership): the more GPUs you buy, the more you save.

In the inference era, Jensen's token economics flip the logic:

AI inference has very healthy gross margins, so the logic now becomes: the NVIDIA GPU is the GPU that produces the cheapest token in the world, so the more you buy, the more you earn.

The top-level KPI has become a Pareto frontier: along the two dimensions of token throughput and token speed, optimize as far as possible.

Each generation of NVIDIA's token factory is essentially pushing the entire Pareto frontier up and to the right. This is the most important KPI of the AI inference era.

———————————————————————————————

From Token Throughput to HBM: The Core Logic Chain

Below is the most important logical chain of this article: how to start from the exponential growth of token throughput and derive that the ceiling bottleneck lies in the exponential growth of HBM size and HBM speed.

In the era of single-GPU inference with single-thread batch size = 1, token throughput had only one dimension: HBM bandwidth speed. Higher bandwidth = higher token throughput.

But once we entered the NVL72 era, inference is no longer single-GPU. It is a system-level token factory composed of 72 GPUs + 36 CPUs, designed to fully saturate HBM bandwidth and compute simultaneously, in pursuit of the ultimate token throughput.

Token throughput growth depends on two things: the number of requests batched simultaneously × the average token speed per request.

That is: batch size × token speed.

Take Rubin NVL72 as an example. At an average token speed of 100 tokens/s, processing 1,920 simultaneous requests yields a token throughput of 192,000 tokens/s. A Rubin NVL72 draws roughly 120kW (0.12MW), so per MW it can handle 1.6M tokens/s.

So we need to find ways to push both parameters up: batch size and average token speed. Their product is our top-level KPI — token throughput.

Parameter 1: Batch growth — bottleneck is HBM size

Every request in the batch carries its own KV cache, which has to live in HBM, with sizes ranging from a few GB to tens of GB. Because hot KV cache must be read at high frequency and high speed at any moment, it must reside in HBM. For a model with, say, 80 layers, every token generation step requires reading the KV cache 80 times from HBM.

As batch size grows, hot KV cache grows linearly.

And because the hot KV cache for every request in the batch must sit in HBM, HBM size must grow linearly with batch size.

Like an airport shuttle bus: the gate wants to move passengers to the plane as fast as possible. If HBM size is small, the shuttle is small, so you have to make extra trips.

Conclusion: batch size growth bottlenecks on HBM size growth.

Parameter 2: Average token speed per request — bottleneck is HBM bandwidth

The decode-phase speed of a large model bottlenecks on HBM bandwidth, because every token generated requires reading the activated weights and KV cache many times over.

The emergence of LPUs has, in cases where batch size isn't very large, moved the activated weights portion onto SRAM — but every generated token still requires many reads of the KV cache from HBM. The higher the HBM bandwidth, the faster each token is generated, in essentially linear correspondence.

Like the airport shuttle bus: HBM bandwidth is like the width of the door — wider doors mean passengers board faster.

The rest of the GPU's configuration is essentially adapted to support batch growth and to keep token compute speed in step with HBM growth. In some cases the GPU even spends excess compute to recover effective bandwidth (e.g., bandwidth compression techniques).

—-------

To return to the shuttle bus analogy:

• Shuttle bus cabin size = HBM Size (capacity): determines how many passengers can fit at once (i.e., how many requests' KV caches can sit in HBM simultaneously). Bigger cabin = more passengers (higher batch size) per trip. If the bus is too small, moving 100 people takes two trips — and total throughput suffers.

• Shuttle bus door width = HBM Bandwidth: determines how fast passengers get on and off. A wide door, and everyone piles on at once (decode/token generation is fast). A narrow door, and even with a giant cabin, people queue up and most of the time is spent boarding.

• Passenger throughput = cabin size × door-width-determined boarding speed.

—-------

At this point, we've logically derived the first principle of token-economics hardware demand:

Token throughput = HBM size × HBM Bandwidth

The top-level KPI of the AI inference era is highly dependent on progress along both HBM dimensions.

If we want to maintain 2× token throughput growth per generation, that means each generation of single GPU must grow HBM size × HBM BW speed by 2×!

This is the first time in history that HBM memory size can influence the top-level KPI — token throughput.

To validate this thesis, we can put NVIDIA's token throughput from A100 to Rubin Ultra on the same chart as HBM size × HBM BW speed.

What you find is that the two curves track each other startlingly closely on log axes.

HBM size × speed actually grows even faster than token throughput — which makes sense, because HBM defines the ceiling, and in practice utilization of that ceiling is very hard to push to 100%. Even if HBM size × HBM speed grew by 1,000×, with the supporting compute and architecture, it would be very hard to wring out the full 1,000× of headroom.

This curve isn't a coincidence — it's the necessary solution of system optimization.

throughput = batch × speed. This is the unavoidable first principle of token factory economics.

—-------

What about software? Won't software optimization reduce bandwidth demand? Reduce HBM demand?

This is an independent dimension from hardware. It's like asking: if software on a CPU runs faster after optimization, does that mean the CPU doesn't need to advance for ten years? After all, software is faster now.

If that were the case, would CPU vendors still make money? For a CPU vendor to survive, there's only one path: in standardized benchmarks, ignoring software optimization, every new CPU generation must score higher — otherwise it doesn't sell.

GPUs are exactly the same. How well software is optimized, and the requirement that the GPU's own token-throughput KPI must improve dramatically every year, are two separate things.

As long as token demand keeps growing, the pursuit of higher token throughput will not stop — and so neither will the pursuit of higher HBM size × HBM speed.

If HBM size and HBM speed were to slow down, Jensen would personally fly to the Big Three and pressure them to accelerate, because that ishis GPU ceiling. If the ceiling stops rising, can his GPU still sell?

Of course, NVIDIA also needs to wrack its brains to extract performance beyond the HBM ceiling through heterogeneous architectural angles. The LPU is a great example — it improved the Pareto frontier substantially from a different angle (the right-hand high-token-speed portion).

—--------------------

HBM memory has now bid farewell to that old era of drifting with the tide. On this one-way road paved by exponential demand, it has, in something close to a destined fashion, walked onto the central stage of the industry's epic.

When the inference paradigm's first principles evolve to this point, as long as Jensen still wants to sell GPUs, HBM must double — and it must double every generation. This is endogenous pressure from the supply side. It has nothing to do with AI demand, nothing to do with macro cycles, and nothing to do with the moods of the hyperscalers.

The only remaining question is this:

When demand has been physically locked into exponential growth, will the three players on the supply side — like they have for the past thirty years — once again drag themselves back into the mire of the cycle by their own hands?

[Ihtesham Ali @ihteshamali]

A mathematician who shared an office with Claude Shannon at Bell Labs gave one lecture in 1986 that explains why some people win Nobel Prizes and other equally smart people spend their whole lives doing forgettable work.

His name was Richard Hamming. He won the Turing Award. He invented error-correcting codes that made modern computing possible. And he spent 30 years at Bell Labs sitting in a cafeteria at lunch watching which scientists became legendary and which ones faded into nothing.

In March 1986, he walked into a Bellcore auditorium in front of 200 researchers and told them exactly what he had seen.

Here's the framework that has been quoted by every serious scientist for the last 40 years.

His opening line landed like a punch. He said most scientists he worked with at Bell Labs were just as smart as the Nobel Prize winners. Just as hardworking. Just as credentialed. And yet at the end of a 40-year career, one group had changed entire fields and the other group was forgotten by the time they retired.

He wanted to know what the difference actually was. And he said it wasn't luck. It wasn't IQ. It was a specific set of habits that almost nobody is willing to follow.

The first habit was the one that hurts the most to hear. He said most scientists deliberately avoid the most important problem in their field because the odds of failure are too high. They pick a safe adjacent problem, solve it cleanly, publish it, and move on. And because they never swing at the hard problem, they never hit it. He said if you do not work on an important problem, it is unlikely you will do important work. That is not a motivational line. That is a logical one.

The second habit was about doors. Literal doors. He noticed that the scientists at Bell Labs who kept their office doors closed got more done in the short term because they had no interruptions. But the scientists who kept their doors open got more done over a career. The open-door scientists were interrupted constantly. They also absorbed every new idea passing through the hallway. Ten years in, they were working on problems the closed-door scientists did not even know existed.

The third habit was inversion. When Bell Labs refused to give him the team of programmers he wanted, Hamming sat with the rejection for weeks. Then he flipped the question. Instead of asking for programmers to write the programs, he asked why machines could not write the programs themselves. That single inversion pushed him into the frontier of computer science. He said the pattern repeats everywhere. What looks like a defect, if you flip it correctly, becomes the exact thing that pushes you ahead of everyone else.

The fourth habit was the one that hit me the hardest. He said knowledge and productivity compound like interest. Someone who works 10 percent harder than you does not produce 10 percent more over a career. They produce twice as much. The gap doesn't add. It multiplies. And it compounds silently for years before anyone notices.

He finished the lecture with a line I have never been able to shake.

He said Pasteur's famous quote is right. Luck favors the prepared mind. But he meant it literally. You don't hope for luck. You engineer the conditions where luck can land on you. Open doors. Important problems. Inverted questions. Compounded hours. Those are not traits. Those are choices you make every single day.

The transcript has been sitting on the University of Virginia's computer science website for almost 30 years. The video is free on YouTube. Stripe Press reprinted the full lectures as a book in 2020 and Bret Victor wrote the foreword.

Hamming died in 1998. He gave his final lecture a few weeks before. He was 82.

The lecture that explains why some careers become legendary and others disappear is still free. Most people who could benefit from it will never open it.

[Elon Musk @elonmusk]

Starship is the most powerful moving object ever made

[Allen Braden @allen_explains]

This 2-hour Stanford lecture breaks down how models like ChatGPT and Claude are actually built, clearer than what many people in top AI roles ever get exposed to.

Save this and set aside two hours today. It might end up being the most valuable thing you learn all week.

[Gary Marcus @GaryMarcus]

Claude Code is not AGI, but it is the single biggest advance in AI since the LLM.

But the thing is, Claude Code is NOT a pure LLM. And it’s not pure deep learning. Not even close.

And that changes everything.

The source code leak proves it. Tucked away at its center is a 3,167 line kernel called print.ts.

print.ts is a pattern matching. And pattern matching is supposed to be the *strength* of LLMs.

But Anthropic figured out that if you really need to get your patterns right, you can’t trust a pure LLM. They are too probabilistic. And too erratic.

Instead, the way Anthropic built that kernel is straight out of classical symbolic AI. For example, it is in large part a big IF-THEN conditional, with 486 branch points and 12 levels of nesting — all inside a deterministic, symbolic loop that the real godfathers of AI, people like John McCarthy and Marvin Minsky and Herb Simon, would have instantly recognized.*

Putting things differently, Anthropic, when push came to shove, went exactly where I long said the field needed to go (and where @geoffreyhinton said we didn’t need to go): to Neurosymbolic AI.

That’s right, the biggest advance since the LLM was neurosymbolic. AlphaFold, AlphaEvolve, AlphaProof, and AlphaGeometry are all neurosymbolic, too; so is Code Interpreter; when you are calling code, you are asking symbolic AI do an important part of the work.

Claude Code isn’t better because of scaling.

It’s better because Anthropic accepted the importance of using classical AI techniques alongside neural networks — precisely marriage I have long advocated.

It’s *massive* vindication for me (go see my 2019 debate with Bengio for context, or to my 2001 book, The Algebraic Mind), but it still ain’t perfect, or even close.

What we really need to do to get trustworthy AI rather than the current unpredictable “jagged” mess, is to go in the knowledge-, reasoning-, and world-model driven direction I laid out in 2020, in an article called the Next Decade in AI, in which neurosymbolic AI is just the *starting point* in a longer journey.*

Read that article if you want to know what else we need to do next.

The first part has already come to pass. In time, other three will, too.

Meanwhile, the implications for the allocation of capital are pretty massive: smartly adding in bits of symbolic AI can do a lot more than scaling alone, and even Anthropic as now discovered (though they won’t say) scaling is no longer the essence of innovation.

The paradigm has changed.

—
*Claude Code is plainly neurosymbolic but the code part is a mess; as Ernie Davis and I argued in Rebooting AI in 2019, we also need major advances in software engineering. But that’s a story for another day.

[Roan @RohOnChain]

This 2 hour Stanford lecture on AI careers will teach you more about winning in the AI race than every piece of AI content you have scrolled past this year.

Bookmark this & give it 2 hours, no matter what. It'll be the most productive thing you could do this weekend.

[Massimo @Rainmaker1973]

In a stunning breakthrough, electrons in graphene have exhibited behavior long considered impossible by physicists. At the material's Dirac point—a critical electronic state where graphene is neither fully a metal nor an insulator—the electrons cease behaving like individual particles and instead flow collectively as a nearly perfect quantum liquid.

This strange fluid is extraordinarily smooth, with a viscosity so low it rivals the ultra-hot plasma that existed in the early universe or is recreated in modern particle accelerators—far smoother than any known behavior in ordinary solid matter.

The most shocking discovery: heat and electric charge decoupled completely, resulting in the largest violation ever observed of the Wiedemann–Franz law. This fundamental rule, which has held for over a century in all conventional metals, states that heat and electrical conductivity should move in lockstep. In graphene's quantum fluid, however, the ratio deviated by more than 200 times from the expected value.

This makes graphene far more than just a wonder material—it serves as a remarkable laboratory for exploring extreme quantum phenomena once thought observable only in black holes, quark-gluon plasmas, or the conditions inside massive particle colliders.

Beyond its fundamental importance, this ultra-clean, highly responsive quantum behavior could lead to revolutionary applications, including next-generation ultra-sensitive sensors capable of detecting minute electrical or magnetic fields with unprecedented precision.

["Universality in quantum critical flow of charge and heat in ultraclean graphene." Nature Physics, 13 August 2025]

[Fernando @Franc0Fernand0]

One of the best math books I've ever read:

MIT's "Mathematics for Computer Science"

Its writing style is brilliant, and it covers everything:

- Linear algebra
- Series
- Logic
- Probability
- Number theory
- Graphs

You can find the PDF here:

https://t.co/iQvaflkDPD

[Massimo @Rainmaker1973]

Cats have a wide spectrum of vocalizations, from the 'standard' meow, to purrs, to chirps and trills.

Sometimes they produce these lower pitch vocalizations that vaguely sound like a human speech.

[Hui United @HuiUnited]

@BenCrellin Which goalkeeper do you plan to buy in BGW29? Thanks.

[Tim Ferriss @tferriss]

Some Favorite Life Advice for 2025 (links below)

1. “A message for the New Year” from Brandon Stanton (@humansofny), creator of Humans of New York. This relates to something he and I have texted about often: attention. How should you think about protecting your own attention in 2025? If you don’t have attention, the time and money you have matter little. Attention is the mother resource. Brandon’s post was first written earlier this year, but it’s more relevant every day.

2. ​Coach Christopher Sommer’s email to me. Coach Sommer is a former US National Team gymnastics coach and the creator of GymnasticBodies.

3. “Why You Should Seek More Awe in the New Year” by Ashley Stimpson. Every time I’ve done a past-year review (PYR), a pattern emerges: peak positive emotional experiences are correlated to awe at least 70% of the time. For at least the past 3–5 years, this has been so consistent that I often determine what big blocks to schedule in the new year based on potential for awe. The payoffs include time dilation and, more broadly, traversing the miraculous canvas of full human experience. Friends have asked me why I do silent retreats in nature, why I love ski touring, why I hunt once in a blue moon, or why I am deeply interested in psychedelic science and psychedelic-assisted therapies. If I had to sum it all up in one word, it would be: Awe. But what exactly is “awe,” and how can we embrace more of it? I haven’t found a better article exploring these topics than Ashley Stimpson’s “Awestruck,” featured in Johns Hopkins Magazine, so I asked for permission to publish on my blog, which was graciously granted. I hope you find it as thought-provoking as I did.

4. “Masters of Love.” (The Atlantic). This is the best article on courtship, love, and dating that I’ve read in years. It’s not only well-researched and full of memorable insights, it’s very actionable and gives labels to concepts you can use. Don’t let the subheadline throw you; it’s terrible. “Science says lasting relationships come down to—you guessed it—kindness and generosity” would make me think, “Yeah, duh. I’ll skip this one.” But the article is much more interesting and helpful than the subheadline implies.

5. “The Most Important Question of Your Life” by Mark Manson (@markmanson). I could see reading this once a week. Excellent reminders all around.

6. “So you wanna de-bog yourself” by Adam Mastroianni (@a_m_mastroianni). I only just found Adam, but his writing is some of the most hilarious—and helpful—I’ve seen in a long time. Really worth the read. Big thanks to Crazy B for sending this to me.

7. “63 principles” by Nabeel S. Qureshi (@nabeelqu). Nabeel is currently a stealth startup founder but was previously at Palantir. He has been a visiting scholar in AI at the Mercatus Center as well as an Emergent Ventures awardee.

8. “101 Additional Advices” by Kevin Kelly (@kevin2kelly). Regular readers know that I consider Kevin one candidate for the real-world most interesting man in the world.

9. “You can’t hoard life” by Oliver Burkeman (@oliverburkeman), author of Four Thousand Weeks: Time Management for Mortals. Hat tip to Chris Williamson (@ChrisWillx) for sending this to me.

10. “How To Be Successful,” 13 lessons on achieving outlier success written by Sam Altman (@sama), CEO of OpenAI. Here is one of the many paragraphs I highlighted for myself: “It’s useful to focus on adding another zero to whatever you define as your success metric—money, status, impact on the world, or whatever. I am willing to take as much time as needed between projects to find my next thing. But I always want it to be a project that, if successful, will make the rest of my career look like a footnote.”
​
Here is another: “I believe that it’s easier to do a hard startup than an easy startup. People want to be part of something exciting and feel that their work matters. If you are making progress on an important problem, you will have a constant tailwind of people wanting to help you. Let yourself grow more ambitious, and don’t be afraid to work on what you really want to work on.”