Approaching.AI

is a high-performance AI infrastructure company focused on large model inference optimization.

[Approaching.AI @ApproachingAI]

https://t.co/lSgpkevhhO Completes RMB 1.3B+ Series B Financing 📣

💰 Over RMB 1.3 billion raised in this round (approximately US$180 million)
💰 RMB 2.4 billion raised in total in 2026 (approximately US$330 million)
📈 Selected core operating metrics grew by more than 10× over six months

Why is a company focused on AI Token production attracting attention?

1️⃣ AI Tokens are entering a quality-driven era
Enterprises need more than models that can generate responses. Production-grade AI services must also deliver speed, stability, scalability and sustainable unit economics.

2️⃣ Fewer models, deeper optimization
https://t.co/lSgpkevhhO focuses on a select group of models and scenarios with clear production demand. Through continuous optimization across the inference pipeline, the company aims to produce more effective Tokens per unit of resource while meeting business requirements.

3️⃣ Bringing China-developed computing resources into production
Through heterogeneous resource collaboration, cache reuse and inference-pipeline optimization, different resources can contribute where they are best suited—supporting continued improvements in production efficiency, service stability and unit cost.

[Approaching.AI @ApproachingAI]

KTransformers v0.7.1 is here 🚀

Now supporting:
• Qwen VLM multimodal LoRA fine-tuning in BF16
• Native RAWINT4 LoRA for Kimi K2.5 / K2.6

Fine-tune massive VLM and MoE models with a more memory-efficient training path.

[Approaching.AI @ApproachingAI]

🚀 Over one trillion tokens per day, powered by Mooncake and SGLang.

@ApproachingAI shares the system architecture and engineering lessons behind its token factory, running leading trillion-parameter models with SGLang HiCache + Mooncake Store.

A deep dive into:
🏗️ Production architecture and system design trade-offs
⚡ Performance optimizations for trillion-token-scale workloads
🛡️ Failure isolation, fault tolerance, and recovery

Real production lessons on scaling KV cache reuse while meeting strict inference SLOs.

Read more:
https://t.co/yBfQD275k9

[Approaching.AI @ApproachingAI]

🚀 KTransformers v0.7.0 is here!

This release brings major upgrades for ultra-large MoE fine-tuning:
• Native FP8 Expert weights for LoRA, reducing host memory usage for DeepSeek-V3.1 from ~1.4 TB to ~800 GB
• Support AVX-512 backend for compatible AMD and x86 CPUs
• Support full fine-tuning • Checkpoint saving and resuming
• A complete KTransformers × LLaMA-Factory fine-tuning Cookbook
📷 Blog: https://t.co/ImpPbq91Uq

#KTransformers #LLM #FineTuning #DeepSeek

[Approaching.AI @ApproachingAI]

📍 vLLM Community Meetup — May 23

https://t.co/lSgpkevhhO joined the vLLM @vllm_project community meetup themed "How will inference acceleration frameworks reshape LLM deployment in the Agentic AI era?" — alongside Red Hat, Inferact, Alibaba, NVIDIA, Moonshot AI, and the open-source community.

Lu Jiahao (https://t.co/lSgpkevhhO tech expert, Mooncake core contributor) co-presented "llm-d + Mooncake: Large-Scale Agentic AI Inference in Practice" with Greg Pereira, core maintainer of llm-d.

━━━━━━━━━━

🔹 The Agentic AI bottleneck

As context lengths grow, traditional compute paradigms face new cost and performance pressure. But multi-turn interactions, tool calls and long-context reuse give Agentic workloads very high KV cache hit rates — a window for "storage-for-compute" trade-offs.

🔹 Mooncake: store more, transfer fast

Transfer Engine (TENT) — evolves from a passive communication library into an active orchestrator. Declarative orchestrationreplaces imperative APIs, delivering low-latency, high-throughput transfer across heterogeneous interconnects.

Mooncake Store — a high-capacity, low-cost SSD tier joins the cache hierarchy. TCO drops, performance holds, and the KV cache poolscales beyond single-machine limits for cross-instance, cross-node context reuse.

🔹 llm-d × Mooncake roadmap

Phase 1 — MooncakeConnector: PD disaggregation, P2P transfer, Messages API
Phase 2 — MooncakeStoreConnector: tiered cache offload, Responses API

━━━━━━━━━━

https://t.co/lSgpkevhhO will keep contributing to Mooncake and partner with the community to push inference systems toward a new balance of
scale, performance and cost for the Agentic AI era.

[Approaching.AI @ApproachingAI]

Handy for capacity planning and config sizing — give it a try 👍

[KVCache.AI @KVCache_AI]

🚀 We just launched the open-source KV Cache Size Calculator by https://t.co/GavTTEDu5C!

Calculate KV cache size for mainstream LLMs with flexible precision settings and detailed breakdowns.

Supports DeepSeek, GLM, Kimi, Qwen3 and MiniMax.

Try it now: https://t.co/PpN0bvKSVI

[Approaching.AI @ApproachingAI]

At AMD Developer Day @AIatAMD 🚀
Tsinghua University Assoc. Prof. and https://t.co/lSgpkevhhO co-initiator Mingxing Zhang @james0zan gave a talk:
“How Do We Reduce Token Costs in the Agent Era?”
---
The core shift in AI workloads:

We're moving from simple chatbots to complex agent applications.
Each agent task requires dozens of model calls and heavy context I/O, pushing token consumption from 1x to 1000x.
Not all tokens share the same SLO, and compute infrastructure does not equal efficient token production.
Cache, transfer, and token reuse matter as much as the silicon underneath.
---
This thinking shapes Mooncake
https://t.co/jfLpfkN40F

The open-source KV-cache-centric serving substrate for modern LLMs.
Originally focused on KV cache — store more, transfer fast, easy to use — Mooncake has grown into a common
substrate for disaggregated LLM serving:
✅ PD / EPD / RL disaggregation
✅ Large-scale elastic EP
✅ Efficient model weight loading
---
Latest Mooncake highlights from the talk:

⚡ Checkpoint Engine — 1T parameters synced in seconds for distributed RL
⚡ Elastic EP in SGLang — partial failure tolerance for DeepSeek MoE deployments
⚡ vLLM native integration — scaling agentic workloads in production
⚡ Omni Models
⚡ TorchSpec — decoupled speculative decoding framework, streaming hidden states from inference to training
⚡ RL toolchains — TransferQueue / Slime (WIP) / Roll (WIP) for cross-device async non-blocking scheduling
⚡ Production-proven — large-scale deployment validated by RBG + SGLang + Mooncake
---
Thanks @AIatAMD for hosting.
We look forward to deepening collaboration with the AMD developer community on the infrastructure layer of the agent economy.

[Approaching.AI @ApproachingAI]

The goal: let developers work with frontier LLMs on the hardware they already have.Local inference → local fine-tuning. Model adaptation → multi-platform. KTransformers is building more open local LLM infrastructure.

[Approaching.AI @ApproachingAI]

At GOSIM Paris 2026 @gosimfoundation , https://t.co/TNZHDOONnW's Chief Engine Architect Ervin Xie shared what's next for KTransformers: local fine-tuning via KT-SFT (run + tune on one workstation), and broader platform support — including Windows.

[Approaching.AI @ApproachingAI]

5/ Deploy in 3 steps: download weights from HuggingFace → launch SGLang server with KT-Kernel → interact via KT CLI or API. Full guide:
doc/en/DeepSeek-V4-Flash.md

[Approaching.AI @ApproachingAI]

1/ KTransformers now supports DeepSeek-V4 on consumer hardware. Single RTX 5090 + RAM → V4-Flash at 20 tok/s. Two RTX 5090s → V4-Pro. First time a
model at this scale runs end-to-end on consumer GPUs.

[Approaching.AI @ApproachingAI]

4/ New operators: MX_FP4_MOE_TP on CPU (native E2M1 quant + ue8m0 scaling, group_size=32). SM_120 portable MXFP4 MoE kernel on GPU for Blackwell. Full path from quantization format to hardware execution.

[Approaching.AI @ApproachingAI]

Proud to be a launch partner for Kimi K2.6! 🎉
we shipped full support on Day 0：
✅ Token service live；
✅ KTransformers adapted — full-precision inference & fine-tuning with just 48GB VRAM + sufficient RAM
Try it now 👉 https://t.co/D6YauRMrIW

[Kimi.ai @Kimi_Moonshot]

Meet Kimi K2.6: Advancing Open-Source Coding

🔹Open-source SOTA on HLE w/ tools (54.0), SWE-Bench Pro (58.6), SWE-bench Multilingual (76.7), BrowseComp (83.2), Toolathlon (50.0), Charxiv w/ python(86.7), Math Vision w/ python (93.2)

What's new:
🔹Long-horizon coding - 4,000+ tool calls, over 12 hours of continuous execution, with generalization across languages (Rust, Go, Python) and tasks (frontend, devops, perf optimization).
🔹Motion-rich frontend - Videos in hero sections, WebGL shaders, GSAP + Framer Motion, Three.js 3D.
🔹Agent Swarms, elevated - 300 parallel sub-agents × 4,000 steps per run (up from K2.5's 100 / 1,500). One prompt, 100+ files.
🔹Proactive Agents - K2.6 model powers OpenClaw, Hermes Agent, etc for 24/7 autonomous ops.
🔹Claw Groups (research preview) - bring your own agents, command your friends', bots & humans in the loop.
-
K2.6 is now live on https://t.co/YutVbwktG0 in chat mode and agent mode.
For production-grade coding, pair K2.6 with Kimi Code: https://t.co/uvoSJKyGCY
-
🔗 API: https://t.co/EOZkbOwCN4
🔗 Tech blog: https://t.co/9wWvgIQSS3
🔗 Weights & code: https://t.co/Be0hjs2RTP

[Approaching.AI @ApproachingAI]

Mooncake × SGLang introduce Elastic Expert Parallelism (Elastic EP) for large-scale MoE inference.

With partial failure tolerance, Elastic EP enables recovery from GPU/node failures in seconds — without interrupting serving.
https://t.co/o37uXG2UQ8

[Approaching.AI @ApproachingAI]

Big update from https://t.co/1v5Wy3AVgv:

Dr. Jenny Wu joins as President, bringing experience from Baidu Capital, https://t.co/1l61Wllrja, Morgan Stanley, and New Hope.

She’ll lead strategy, finance, and global ops as we scale efficient AI inference.

More to come.

#Inference