Tensormesh

AI inference optimization company that never charges you twice for cached tokens, making AI applications faster and dramatically cheaper to run anywhere.

[Akshay 🚀 @akshay_pachaar]

https://t.co/3DkJWEFHaJ

[Tensormesh @tensormesh]

AI infrastructure UX is not about giving developers a longer product tour.

It is about helping them deploy a model, test an API, understand pricing, compare latency, and reach a real technical outcome quickly.

Our latest blog, based on an interview with Katherine Yee, Software Engineer at @tensormesh , explores why infrastructure UX has to balance technical depth with clarity.

For context caching, that means showing users when repeated context is reused, how cache hits affect cost and latency, and why metrics like cached input tokens and cost per task matter for AI agents, coding assistants, and long-context workflows.

The best infrastructure UX teaches while developers build.

Full blog: https://t.co/96QseoJUlU

[LMCache Lab @lmcache]

𝐋𝐌𝐂𝐚𝐜𝐡𝐞 𝐧𝐨𝐰 𝐬𝐮𝐩𝐩𝐨𝐫𝐭𝐬 𝐌𝐨𝐨𝐫𝐞 𝐓𝐡𝐫𝐞𝐚𝐝𝐬 𝐆𝐏𝐔𝐬 𝐰𝐢𝐭𝐡 𝐊𝐕 𝐜𝐚𝐜𝐡𝐞 𝐨𝐟𝐟𝐥𝐨𝐚𝐝𝐢𝐧𝐠!

Moore Threads is a GPU hardware vendor, and MUSA is its compute platform for running workloads on Moore Threads GPUs, similar to CUDA for NVIDIA GPUs.

With this update, LMCache continues to expand its multi-platform support across NVIDIA, AMD, Ascend, and Moore Threads.

LMCache can now store and load KV cache to and from MUSA paged memory, with support for both standard attention and MLA attention layouts.

You can get the 𝐯𝐋𝐋𝐌 + 𝐌𝐔𝐒𝐀 + 𝐋𝐌𝐂𝐚𝐜𝐡𝐞 stack ready to use with a simple build flag:

𝘉𝘜𝘐𝘓𝘋_𝘞𝘐𝘛𝘏_𝘔𝘜𝘚𝘈=1 𝘱𝘪𝘱 𝘪𝘯𝘴𝘵𝘢𝘭𝘭 -𝘦 . --𝘯𝘰-𝘣𝘶𝘪𝘭𝘥-𝘪𝘴𝘰𝘭𝘢𝘵𝘪𝘰𝘯

This is a correctness-first implementation. Some features, including layerwise transfer, CacheBlend, SGLang, and more, are not yet supported on MUSA.

If you are building on Moore Threads GPUs and are interested in persistent KV cache storage, we’d love for you to join us in growing MUSA support in the LMCache ecosystem!

#LMCache #vLLM #KVCache #摩尔线程 #MUSA #MooreThreads #OpenSource

[Tensormesh @tensormesh]

KV cache blending helps AI apps avoid recomputing context they have already processed.

Instead of treating every request like a fresh start, cached context from prompts, documents, tools, and histories can be reused more efficiently across workflows.

That is a big deal for agentic applications, long-context use cases, and production inference costs.

[Tensormesh @tensormesh]

What makes the new generation of developers so special in this new AI era? Hear our CEO @JunchenJiang's perspective at @AIatAMD

[AI at AMD @AIatAMD]

The future of software engineering is about more than writing code.

@JunchenJiang, Co-founder of @TensorMesh, shares his perspective on AI and the next generation of developers.

[Tensormesh @tensormesh]

Most AI teams are paying GPU prices to repeatedly process context they’ve already seen.

Same system prompts.
Same tool definitions.
Same knowledge base.
Same instructions.

The cache disappears, so the bill comes back.
Persistent KV cache changes that equation.

New blog on why owning the cache lifecycle may be one of the biggest cost optimization opportunities in AI inference 👇

https://t.co/FylmynJe7e

#AIInference #LLM #KVCache #AIInfrastructure

[LMCache Lab @lmcache]

𝐋𝐌𝐂𝐚𝐜𝐡𝐞 𝐯𝟎.𝟓.𝟎 𝐢𝐬 𝐡𝐞𝐫𝐞!

Release highlights:

𝐏𝐞𝐞𝐫-𝐭𝐨-𝐩𝐞𝐞𝐫 𝐊𝐕 𝐭𝐫𝐚𝐧𝐬𝐟𝐞𝐫 — LMCache instances can now look up and pull KV cache directly from each other in MP mode.

𝐀𝐬𝐲𝐦𝐦𝐞𝐭𝐫𝐢𝐜 𝐬𝐞𝐫𝐢𝐚𝐥𝐢𝐳𝐚𝐭𝐢𝐨𝐧 — Mix precisions in one KV cache (e.g. FP16 keys, FP8 values).

𝐃𝐞𝐯𝐢𝐜𝐞-𝐃𝐀𝐗 𝐋𝟏 𝐨𝐯𝐞𝐫𝐟𝐥𝐨𝐰 — Spill hybrid L1 cache to Device-DAX, plus device-agnostic IPC across GPUs, CPUs, and accelerators.

𝐖𝐨𝐫𝐤𝐞𝐫 𝐥𝐢𝐯𝐞𝐧𝐞𝐬𝐬 𝐭𝐫𝐚𝐜𝐤𝐢𝐧𝐠 — Automatic liveness tracking and reaping for MP servers.

𝐍𝐞𝐰 𝐛𝐚𝐜𝐤𝐞𝐧𝐝𝐬 SGLang XPU connectors, native Aerospike L2, Moore Threads MUSA.

Huge thanks to all contributors! Read more: https://t.co/tmqs2yclf3

#LMCache #KVCache #LLM #AIInfrastructure #OpenSource

[Tensormesh @tensormesh]

LLM responses feel slow and expensive for one big reason: the model recomputes the same attention math on every single token.

KV caching fixes that.

Less compute → lower cost → faster output

[Tensormesh @tensormesh]

Open-weight LLMs can look cheaper on paper, but production cost depends on how often your app reprocesses the same context.

System prompts, tools, docs, codebase context, and histories should not become a recurring tax.

That is the Amnesia Tax.

See how Tensormesh removes it with $0 cached input tokens.

https://t.co/aM39BFdrp5

[Tensormesh @tensormesh]

What does it take to build the next generation of AI infrastructure?
Hear our CEO @JunchenJiang perspective at AMD AI DevDay.

Appreciate the spotlight, @AIatAMD 👏🏻

[AI at AMD @AIatAMD]

The future of AI depends on how compute, storage, networking, and software work together.

@JunchenJiang, Co-founder of @tensormesh, on what it takes to build the next generation of AI infrastructure.

[Tensormesh @tensormesh]

Claude Code can run open-weight LLMs with @tensormesh Serverless Inference.

Three environment variables let you point Claude Code at models like Qwen3-Coder, MiniMax, DeepSeek, and Kimi without managing GPUs, vLLM, or serving infrastructure.

📘Full guide:
https://t.co/5rtuECSB9L

[Tensormesh @tensormesh]

GPU memory will never be big enough, that's why reusing your KV cache instead of recomputing it is one of the biggest cost savers in AI inference.

[Tensormesh @tensormesh]

You can now run open-weight LLMs in Codex CLI with @tensormesh Serverless Inference.

Use models like @MiniMax_AI , Qwen3-Coder, @Kimi_Moonshot, Devstral, and gpt-oss in the same Codex agent loop without a fork, plugin, GPU setup, or local inference server.

Model choice becomes a flag instead of a migration.

We wrote a 3-step guide for getting started in about 5 minutes.

https://t.co/KBq0f3D11r

[Tensormesh @tensormesh]

Deploying 1,000 GPU nodes? You could be overpaying millions per year.

KV Cache optimization changes costs dramatically.

[Tensormesh @tensormesh]

A lot of people still default to closed models because they feel like the safest or highest-quality option, but open-source models are much closer than people realize for the right workloads.

The gap is often the infrastructure around them.

With a platform like Tensormesh, teams can use open models with caching-accelerated inference, cleaner APIs, and better economics around repeated context.

That makes open models much more practical for agent and RAG workloads where the same prompts, tools, docs, and histories show up again and again.

Open models bring the flexibility and the serving layer is what makes that flexibility usable in production.

[Tensormesh @tensormesh]

This is the right lens as longer-running agentic jobs do not just increase token demand, they increase the amount of KV state that needs to be stored, moved, reused, and served efficiently.

More memory supply matters, but the software layer matters too. If agents keep repeating the same context across steps, jobs, and sessions, then the question becomes how much of that KV state can be reused instead of recomputed.

The memory bottleneck and the inference cost bottleneck are becoming the same problem.

[Tensormesh @tensormesh]

“Waste tokens, save time” is the right framing, but production agents need a better version of that tradeoff.

At Tensormesh, we’re focused on the repeated context problem, where agents keep replaying the same tools, docs, policies, and histories across calls.

Saving time should not mean paying to recompute the same context every run.

[Tensormesh @tensormesh]

This is exactly the scaling problem we’re focused on at Tensormesh.

Request volume, context length, and concurrency all compound as teams move from prototype to production, and agent or RAG apps often add another cost driver with repeated context across calls.

The same system prompts, tool definitions, policies, retrieved docs, and histories can get prefilling again and again, which turns repeated context into repeated cost.

Caching that repeated context can drastically change the inference cost curve.

[Jesse Landry @JesseDLandry]

AI inference has a cost problem hiding under the GPU race.

@tensormesh raised additional Seed funding from @AMD, @CoreWeave, and NVentures, bringing total funding to $24.5M, as @JunchenJiang, Yihua Cheng, and Kuntai Du build KV cache reuse infrastructure to reduce repeated computation, latency, and GPU spend.

The next AI infrastructure winners will not just add more compute. They will make intelligence cheaper to run at scale.