Rakesh Cheerla

Technologist & Product Manager

[Rakesh Cheerla @RakeshCheerla]

https://t.co/Lmqy5fvqDI

[Rakesh Cheerla @RakeshCheerla]

AI agents are moving toward jobs that span hours or days. Memory tiering will play a growing role in scaling these workloads.

[Netpreme @netpreme]

AI memory tiering for long-context LLM inference.

“The Scale-Up Memory Tier”:
https://t.co/5MrYJDewon

Growing KV caches put pressure on GPU HBM. Explore Netpreme X-Mem’s pooled DRAM tier on the scale-up fabric.

#KVCache #AIInfrastructure

[Rakesh Cheerla @RakeshCheerla]

RT @zhizhenzhong: The future of AI systems requires compute and memory to scale independently over high-bandwidth fabrics, such that memory…

[Ed Sim @edsim]

💯 The next 10× may come from wasting less, not fabricating more.

A composable memory fabric can make stranded capacity, yes even from older systems, useful to the newest accelerators.

That’s we need a new architecture, MPUs or memory processing units, Netpreme - more AI state close to compute, without hard-wiring today’s model.

https://t.co/kNQ23K6ynt

[LMSYS Org @lmsysorg]

XM-Sparse: Faster Decode with SGLang HiSparse

@netpreme builds memory systems that give GPUs more capacity. Its X-Mem™ uses a high-bandwidth interconnect to fetch offloaded KV faster than host DRAM over PCIe, with more capacity for larger batches.

- Faster decode: 49.25% / 63.58% lower time per token at 20% / 40% KV misses, versus host DRAM. Same 16K context and batch 192/GPU.
- Higher throughput: 2.09–3.04× host DRAM across 16K–1M contexts at 20% misses, using each setup’s best tested batch.

Read the full blog below 👇

[Rakesh Cheerla @RakeshCheerla]

We may get a rare IPO discount on Anthropic.

Its training and inference efficiencies are quite amazing and cut compute needs.

As industries mature, profits shift from infra (semis) to customer-facing AI platforms.

[Jim Cramer @jimcramer]

i think that Anthropic is going to have to take a discount to get done. They have been the principal progenitor of fear, which is what the American people feel.

[Rakesh Cheerla @RakeshCheerla]

Research Paper Links:
Smarter token selection — SAS : https://t.co/Aj5nXaNu2Q
Hierarchical search — Latent Sparse Attention (LSA): https://t.co/ZeVfh3kkPf
Index reuse — GLM-5.2 / IndexShare: https://t.co/3wFoITpeSp
Hybrid attention — Step 5 Preview: https://t.co/HNuGbFi4NW

[Rakesh Cheerla @RakeshCheerla]

Sparse attention is seeing rapid growth on Open Router. The sparse attention tokens are accounting for 41% of the total portion of tokens, and growing rapidly.

More importantly, research is driving further sparse attention advancements. A few examples
🔹 Smarter selection: SAS learns which context to retrieve.
🔹Hierarchical search: LSA and ZHA find relevant tokens efficiently.
🔹Reuse: CSA2 shares KV caches; IndexShare reuses selections across layers.
🔹Hybrid attention: Qwen3.8-Flash-Next and Step 5 combine sparse retrieval with linear or local attention.

Agents accumulate long histories. Video creates enormous token sequences. Each step often needs only a small subset. The relevance for these usecases and for long context is massive. The memory crunch just makes the need for sparse attention existential.

The hardware must adapt: a large warm memory tier stores the full KV context, while HBM caches the hot tokens. Making this work requires low latency prefetching that overlaps data movement with computation.

Our bet at Netpreme: future AI systems scale memory capacity independently of HBM bandwidth, tiering sparse data into the HBM at scale-up bandwidths that are 10-15x faster than PCIe/CXL bandwidth.

[Rakesh Cheerla @RakeshCheerla]

Congrats Dan, Rohan, Nikhil 🚀

[Nooks @NooksHQ]

Today we’re announcing $500M in growth capital from
@generalcatalyst 🚀

[Andrej Karpathy @karpathy]

Weekend hack: HackerTube. I was annoyed with YouTube's recommendations so I had Claude filter down 43M comments over 20 years of HN to 6,454 videos that keep being recommended. *Significantly* better viewing experience, ~2 hours with Opus 5.5:
https://t.co/JIhXN7OSh1

- Excellent recs without any logins (state is kept in local browser session)
- Thumbnail is a representative frame from middle of the video instead of the super duper A/B test hyper-optimized user uploads
- Underlying data from the comment analysis as one zip file, point your Claude at it and ask for your own version
- HackerTube Longs ✊

A bit more on the process:
I use Claude Code (desktop app for Mac), Opus 5.5 on Medium effort as my default. I only talk to Claude and have it do everything, the only "manual work" I helped with in tandem was in the browser (reminiscent of MenuGen): 1) I had to create some kind of API key from Google Cloud UI to query YouTube for video metadata, 2) I booted up a VPS box for serving and 3) I bought the domain name. Once the box was up I gave Claude the IP and told it update, secure, harden, deploy and optimize the code, and it did the whole thing.

Many awesome projects that would have normally taken weeks/months even can now be few-shotted for breakfast. What else are we not thinking of.

[SemiAnalysis @SemiAnalysis_]

GDPx covers the following domains. Each was a collaboration with at least one specialized data partner. SemiAnalysis was responsible for final QA and heavily involved with task iteration. (2/7)

[Gwynne Shotwell @Gwynne_Shotwell]

Robert and I are thrilled that our gift of over 2 million shares of SpaceX stock will be distributed to children’s Trump accounts in December!

Our initial gift (shares transferring to escrow as we speak) will go to 11 to 17-year-olds who live in zip codes with median family income (MFI) of $57,000 or less, and to those who live in zip codes with too few residents for an MFI to be reported in the census. These families should receive a notice by Monday. 

In a few weeks, an additional gift will be finalized for eligible 11 to 17-year-old children in military families serving overseas, children in foster care, and more children whose MFI is $90,000 or less living in counties near our home in the great state of Texas.

Thank you to the incredible team @SpaceX, @Treasury, @IRS, @DellFdn, @GreaterHorizons, @InvestAmerica24, and the legal firms of Morgan Lewis and Gibson Dunn for working over the past few months to make this happen.

We are deeply grateful for the groundbreaking work that Brad Gerstner @altcap and @MichaelDell have done for the long-term financial security for millions of kids.

God Bless America!

[Rakesh Cheerla @RakeshCheerla]

@thdxr Anthropic spent just $12B on pre- and post-training compute. Google, Meta, SpaceX, Amazon or Nvidia could readily spend $50B. The bottleneck appears to be skilled humans, not capital.

[Netpreme @netpreme]

Bringing real-world AI challenges to Berkeley.

Netpreme × Haas Business Student Association (HBSA): a Product Management Case Competition connecting AI infrastructure with the next generation of product talent.

Register by Nov 6: https://t.co/OGiu8WD3JW

[Rakesh Cheerla @RakeshCheerla]

@MaxAnderson The miss indicates that most high-velue extration options are exhausted.

We enter the post-google search era!

[Rakesh Cheerla @RakeshCheerla]

What must be true for long-run full AI diffusion?

1. Early adopters today have high willingness to pay (WTP) and late adopters want much lower WTP. Jevons and full AI diffusion only kicks in below the next cohort’s WTP threshold.
2. Customers must make more money (revenue, not profits) than the hardware company. Otherwise the pyramid scheme collapses down under its own weight.
3. Pricing power forms at the customer surface aka platforms close to customers. When AI diffuses, profit pools move to the intelligence platforms, currently Anthropic and OpenAI, but that is likely to change, if history is a guide.

Infra firms always capture outsized profits during the buildout. But an ecosystem where hardware suppliers vastly outearn downstream customers - is unstable and temporary.

Perhaps it just means we are early in our buildout!

[Netpreme @netpreme]

Reading less doesn’t mean storing less.

A short look at sparse attention, KV cache, and how Netpreme X-Mem™ brings more capacity and faster access to SGLang HiSparse.

Full article: https://t.co/3ft1Ehmimu

#SGLang #KVCache

[Netpreme @netpreme]

An AI agent pauses for a tool call. Why keep its KV cache in expensive HBM?

Our OCP-week panel explored memory tiering, shared agent state and the cost of completing a task.

https://t.co/C2pbzzNFbC

#AgenticAI #KVCache