Mr. Bojangles

[Mr. Bojangles @crossXhurricane]

@Rafeboogs Stormin Norman

[Mr. Bojangles @crossXhurricane]

@jacksdiningroom Electric burrito

[Rohan Paul @rohanpaul_ai]

New Google paper says LLM inference is limited by memory and networking, and it proposes 4 hardware shifts.

Inference is separated into Prefill, which processes input in parallel, and Decode, which outputs 1 token (a small text chunk) at a time while constantly reading a Key Value cache, a running memory of past text.

LLM serving speed and cost problems come from the Decode phase, where the system keeps fetching past context from memory and talking across chips, not from a lack of raw compute.

The paper argues today’s data center accelerators were shaped by training needs, so they often look fast on paper but waste time during Decode because memory bandwidth, memory capacity, and interconnect latency become the real bottlenecks.

The authors review today’s hardware and show a mismatch, high compute helps less when Decode gets stuck on memory bandwidth, memory capacity, and network latency.

High Bandwidth Flash stacks flash to aim for about 10X more capacity for model weights, the learned numbers that stay fixed during inference, or for slow changing context, and processing near memory plus 3D stacking move compute closer to memory so less energy and time are wasted moving data around.

For fast user responses, the paper also pushes low latency interconnects, meaning fewer network hops and smarter switches, so the same LLM can run on a smaller cluster that costs less and fails less.

----

Paper Link – arxiv. org/abs/2601.05047

Paper Title: "Challenges and Research Directions for LLM Inference Hardware"

[friendscallmeJay @Jaypotta]

Good morning. Here’s Bob Weir performing “Morning Dew” on stage for the 408th and final time, featuring Sturgill MF’n Simpson no less.🔥

[Mr. Bojangles @crossXhurricane]

@WhaleEverything @NoLimitGains @grok DONE

[david friedberg @friedberg]

why not just raise income tax rates?

because your real intent is not to just “provide healthcare”.

you’re masking that you are proposing the creation of, for the first time in the 250 years of this American republic, an organized government seizure of private property from citizens.

you’re calling it a “wealth tax” or a “billionaires tax” or “millionaires tax” or whatever nom du jour polls well. but at the end of the day, it’s the seizure of private property from citizens by the government. citizens that earned money, paid their fair taxes on those earnings (53% if they live in California) and are now being told they need to hand over after-tax assets because the government has failed to provide promised services with the revenue it’s collected, and are now re-casting their own failure to be a socio-economic inequity that must be justly resolved... a slippery slope that has never gone anywhere good (see economic effects in USSR, Cuba, Venezuela, France and Norway wealth tax etc.)

the American founders fled tyranny in Europe and this amazing nation was populated by immigrants (myself and your parents) from around the world not just looking for a “better life” but for a place where they could have freedom from tyrannical governments that can take what they want from private citizens. a great nation borne of property rights, the rule of law, and endowed freedoms to believe, speak, or act. these principles led to the greatest run of innovations, successes, and widespread increase in prosperity, for all citizens, ever seen.

the citizens, the individuals, not the institutions, delivered this progress. those who invented, who toiled, who bled, who sacrificed, who took risk and persevered, who led, and who changed the world, are not charlatans, kleptocrats, or oligarchs. they’re what made us all better off. prosperity is a measure of america’s success, not its failure.

it is your principle that is so offensive, as evidenced by the broad disdain for your flippant flirtation with the darkest of human fantasy - socialism. you and other neo-socialists have led so many of us to reflect on America’s history and what it is becoming. that now leads so many to consider, so unnecessarily, leaving their homes for a place where everyone stands up to shout down the principle you suggest. because if your ideas are now considered moderate, it’s clear this titanic is sinking.

that a “simple tax” of taking assets that have been earned, through toil and tribulation, rightly taxed, and preserved, should now be unjustly seized, is your solution to a problem of obvious government mismanagement and outright fraud, tells us that your true motivation lies not in giving people healthcare but in cutting down success and deleting the system of prosperity and opportunity for all.

i don’t care, and neither should anyone else, what the sum total market value of a private citizens private assets might be. it is none of my business and should be none of yours. because, again, once you open that pandora’s box, we might as well study Lord of the Flies … there is literally nothing stopping 51% of citizens demanding that their government go out and seize 100% of the private property of the 49%.

want to give healthcare to people in need? do your job and fix healthcare. make it affordable. want to be lazy about it? then do your job lazily and raise income taxes.

want to take private property from private citizens who have paid their fair share of taxes and legally earned their property, then honestly declare that it is envy, not inequity, that you strive to resolve…

[Dr. Alex Wissner-Gross @alexwg]

https://t.co/rFoDj5Odjn

[Mr. Bojangles @crossXhurricane]

@Swaggychen51 @grok is this true?

[Mr. Bojangles @crossXhurricane]

@Swaggychen51 LeBron would be nothing without Windy.

[Mr. Bojangles @crossXhurricane]

@LuxuryWatchGuy1 @Stablos_store $515k

[Mr. Bojangles @crossXhurricane]

@GracePersephone @AndrewWhitworth ?

[Mr. Bojangles @crossXhurricane]

@AdamCrigler

[Mr. Bojangles @crossXhurricane]

@Blutman27 @TimJDillon

[Mr. Bojangles @crossXhurricane]

@sophieeeekagain @nickturani @thejeffvibbert That’s John Lennon

[Mr. Bojangles @crossXhurricane]

@Swaggychen51 Buzzer!!

[Mr. Bojangles @crossXhurricane]

@PFTCommenter Yeah but the dude looks like Hank

[Mr. Bojangles @crossXhurricane]

@PFTCommenter That’s Stephen Jackson

[Mr. Bojangles @crossXhurricane]

@PardonMySteak @PardonMyTake @PFTCommenter @hen_ease @Philly_Mays @BarstoolBigCat The AWLs, and Commanders

[Mr. Bojangles @crossXhurricane]

@DonnieDoesWorld Boys of Summer