Netpreme

Building AI-native memory systems, from silicon to tokens. X-Mem™ Networked Memory Tier™.

[Netpreme @netpreme]

AI memory tiering for long-context LLM inference.

“The Scale-Up Memory Tier”:
https://t.co/5MrYJDewon

Growing KV caches put pressure on GPU HBM. Explore Netpreme X-Mem’s pooled DRAM tier on the scale-up fabric.

#KVCache #AIInfrastructure

[Netpreme @netpreme]

Our XM-Sparse article explores how Netpreme X-Mem™ integrates with SGLang HiSparse to provide high-bandwidth access to offloaded KV caches.

Read it here:
https://t.co/0kcFofPb3g

Sparse attention is changing what AI systems need from memory.

DeepSeek and https://t.co/SxSWkdvtRd’s combined share of OpenRouter’s publicly reported tokens rose from 23% in July to 41% in October.*

Meanwhile, research is advancing on several fronts:

🔹 Smarter selection of relevant context
🔹 Hierarchical search for efficient token retrieval
🔹 Reuse of KV data and token selections across layers
🔹 Hybrid architectures combining sparse retrieval with local or linear attention

Agents accumulate long histories. Video creates enormous token sequences. Yet each decode step often needs only a small subset of that context.

The hardware opportunity follows: retain KV history in a large warm memory tier, keep the active working set in HBM, and fetch selected data over a high-bandwidth fabric.

At Netpreme, we’re building toward memory capacity that scales independently of compute, with scale-up bandwidth connecting GPUs to a shared warm memory tier.

*The chart uses DeepSeek + https://t.co/SxSWkdvtRd lab totals as a proxy for sparse-attention adoption, not a model-by-model count. Source: OpenRouter, via CrbonFree: https://t.co/WwiltQkUwC

[Netpreme @netpreme]

Bringing real-world AI challenges to Berkeley.

Netpreme × Haas Business Student Association (HBSA): a Product Management Case Competition connecting AI infrastructure with the next generation of product talent.

Register by Nov 6: https://t.co/OGiu8WD3JW

[Netpreme @netpreme]

For agentic AI, tokens per second tell only part of the story.

At our OCP-week panel, we discussed completed tasks, end-to-end response time and cost per task as measures of AI inference performance.

Read the recap below.

#AgenticAI #AIInfrastructure

[Netpreme @netpreme]

An AI agent pauses for a tool call. Why keep its KV cache in expensive HBM?

Our OCP-week panel explored memory tiering, shared agent state and the cost of completing a task.

https://t.co/C2pbzzNFbC

#AgenticAI #KVCache

[Netpreme @netpreme]

An AI agent pauses for a tool call. Why keep its KV cache in expensive HBM?

Our OCP-week panel explored memory tiering, shared agent state and the cost of completing a task.

https://t.co/C2pbzzNFbC

#AgenticAI #KVCache

[Netpreme @netpreme]

KV cache offloading makes data movement part of decode latency.

Beyond GPU memory, bandwidth and capacity still matter. Our XM-Sparse Pilot VM testbed explores this with SGLang HiSparse for LLM inference.

#KVCache #LLMInference

[Netpreme @netpreme]

Reading less doesn’t mean storing less.

A short look at sparse attention, KV cache, and how Netpreme X-Mem™ brings more capacity and faster access to SGLang HiSparse.

Full article: https://t.co/3ft1Ehmimu

#SGLang #KVCache

[Netpreme @netpreme]

Reading less doesn’t mean storing less.

A short look at sparse attention, KV cache, and how Netpreme X-Mem™ brings more capacity and faster access to SGLang HiSparse.

Full article: https://t.co/3ft1Ehmimu

#SGLang #KVCache

[Netpreme @netpreme]

Sparse attention reads less KV data. The full KV cache still needs fast, high-capacity memory.

Our XM-Sparse Pilot VM testbed with SGLang HiSparse + Netpreme X-Mem™ shows why bandwidth and capacity matter for LLM inference.

See the benchmarks.
https://t.co/s78UcjygM4

[Netpreme @netpreme]

Sparse attention still needs the full KV cache.

Our XM-Sparse Pilot VM testbed with SGLang HiSparse and Netpreme X-Mem™ shows lower TPOT and higher peak decode throughput vs. host DRAM for LLM inference.

https://t.co/6vCTVsdkLs

#LLMInference #SGLang

[Netpreme @netpreme]

Cracking the Memory Wall at #OCPSummit26: a lively discussion on AI inference, KV cache, and memory tiers. Thanks to Hanrui Wang (@hanrui_w), Sudeep Bhoja, Angela Yeung, Kartik Srinivasan, and Rakesh Cheerla (@RakeshCheerla) for joining us—and to Bob Brennan for moderating. #AIInfrastructure

[Netpreme @netpreme]

#OCPSummit26: AI memory wall panel + happy hour.

Bob Brennan moderates Ryan Wang (@hanrui_w, @nebiusai), Sudeep Bhoja (@dMatrix_AI), Angela Yeung (@cerebras), Kartik Srinivasan (Napatech) & Rakesh Cheerla (@RakeshCheerla, @netpreme ).

Oct 14, 5:30–8:30 pm. RSVP: https://t.co/3ZpaBM1WwQ

[Netpreme @netpreme]

Speaker update: Angela Yeung, SVP and Head of Product at @cerebras, joins our Oct 14 AI memory wall panel with Ryan Wang (Nebius), Sudeep Bhoja (d-Matrix), and Rakesh Cheerla (Netpreme). Bob Brennan moderates. Request a spot: https://t.co/AnlPtAGOHZ

[Netpreme @netpreme]

Heading to #OCPSummit26? Join Netpreme on Oct 14 from 5:30–8:30 pm for a panel and happy hour on the AI memory wall. Bob Brennan moderates a discussion with Ryan Wang (Nebius), Rakesh Cheerla (Netpreme) & others TBA. Space is limited. Request a spot: https://t.co/AnlPtAGOHZ

[Netpreme @netpreme]

Speaker update: Sudeep Bhoja, co-founder and CTO of @dMatrix_AI, is joining our AI memory wall panel on Oct 14. We’re looking forward to hearing his perspective alongside Ryan Wang and Rakesh Cheerla.

[Netpreme @netpreme]

@RakeshCheerla Looking forward to seeing everyone! We’ll start with a panel on the AI memory wall, then continue the conversation over food and drinks. If you’d like to join us, request a spot through the Luma link above.