小栗说车日常记

后备箱的隔板设计，影响装载的灵活性｜购车分享 @mothio1201

[PonBroke @BrokePonyal]

AFRICA JUST BECAME THE FIRST CONTINENT TO GET THE NEW TESLA ROADSTER.
NO SHOWROOM. A SHIPPING CONTAINER AND A STEEL RAMP.

Tesla has almost no official stores across Africa.
That never stopped anyone.

The car rolls out of the container backwards.
The new owner steers it down 2 metal rails.
America reserved it first.
Somebody else might drive it first.

Made with AI. The real first delivery is due in 2028.

[小栗说车日常记 @kerdede]

@Dell @McLarenF1 今天只負責假裝自己在專心地看不同的位置 根本無法節省力氣

[Felicia @heyliccia]

this is f**king insane.

this 23 year old used Claude to reverse engineer a kids channel with 28.5M subscribers.

then reportedly made $10K from YouTube Shorts.

he didn't copy individual videos.
he studied the formula behind them:

→ recurring characters

→ simple stories

→ bright visuals

→ fast pacing

→ formats already proven to generate massive views

then he built an AI production engine around those patterns.

Claude analyzes the channel, identifies repeatable ideas, and writes original scripts.

connected AI tools handle the rest:

→ characters
→ voices
→ music
→ animation
→ final assembly

his involvement?

almost nothing.

he chooses the direction, approves the strongest concepts, and lets the system handle the repetitive production.

one prompt → one finished Short.
and because the workflow is reusable, he can keep producing without rebuilding everything from scratch.

the original channel reached 28.5M subscribers across 798 uploads.
he didn't copy one cartoon.

he reverse engineered the machine that could produce an entire channel.
that's the real AI advantage.

[Daily Dose of Data Science @DailyDoseOfDS_]

How do LLMs handle GBs of KV cache in production?

(a popular technical LLM interview question)

At scale, KV cache becomes a storage system.

Every attention layer creates KV tensors while processing a prompt.

During generation, the model reads this state instead of processing the entire sequence again for every new token.

For a single request, this is temporary GPU data.

But when multiple requests and inference workers reuse the same prompt prefix, the cached data must remain available after the original request finishes.

It may even need to survive after the engine that created it restarts.

So a cache hit now requires more than identifying matching tokens.

The system must find the relevant blocks, bring them back into GPU memory, remove old blocks when storage fills, and let multiple workers reuse the same data.

Consider a request that shares 80% of its prompt with an earlier request.

The matching tensors may already be in GPU memory. Otherwise, the system could retrieve them from CPU memory, a local SSD, or remote storage.

The model then processes only the uncached 20%. It stores the newly created tensors so later requests can reuse them.

This involves moving large amounts of data between memory and storage while the GPU is also generating tokens.

Inference engines such as vLLM are mainly designed to schedule requests and run the model. Cache management requires a different set of operations around storage, movement, and cleanup.

The solution is actually open-source and implemented in the LMCache package.

It runs KV cache management as a standalone service beside the inference engine.

The engine requests the required blocks, while LMCache finds them, moves them into the right memory, and stores new blocks afterward.

LMCache uses CUDA IPC, a mechanism that lets separate processes access the same GPU memory. This avoids copying the full cached tensors through ordinary messages between the two processes.

Cached blocks can also move between GPU memory, CPU memory, local storage, and remote storage. On a hit, the inference engine restores the reusable blocks and processes only the missing tokens.

Multiple vLLM instances on the same machine can share one LMCache service. Cache storage can scale separately, and restarting an inference worker does not automatically remove the saved cache.

The LMCache paper reports up to 15x higher throughput when combining it with vLLM across the evaluated workloads.

The visual below traces one prompt through the inference engine, cache service, shared GPU memory, and storage levels.

LMCache GitHub: https://t.co/VCPBztdMhP

(don’t forget to star it ⭐)

If you want to dive deeper, we wrote a detailed breakdown of KV, prefix, prompt, and semantic caching, including where LMCache fits into the serving stack.

Read it below.

[小栗说车日常记 @kerdede]

@JojoBaybeee 我160出头，双脚能前掌着地，坐高不算压人；你平时骑过别的车吗，可以约琼海一起试试～

[小栗说车日常记 @kerdede]

三亚今天骑钱江复古车，像文艺轻熟女，外表温柔，发动以后又藏着一点小脾气。海口、儋州、琼海想看日常骑行体验，主页继续。

[小栗说车日常记 @kerdede]

@migoooooo 海口复古车挺合适，太阳晒一天主要晒的是人，车只要别停着暴晒就行。你想先看通勤还是周末跑线？

[小栗说车日常记 @kerdede]

@gtd1988 起步那下是真的有点响，大爷回头我就冲他笑，反正复古车好认，你在儋州骑的是同一个颜色吗

[小栗说车日常记 @kerdede]

@xSmiley_ox 被发现了……停下拍照时温柔，绿灯一亮就想试探一下，我这算不算人格分裂哈哈

[小栗说车日常记 @kerdede]

@immastar08 博后村那次是顺手的事，阿婆芒果滚一地我也急啊，温柔留给车和人，油门留给路。你也常去博后村？

[小栗说车日常记 @kerdede]

@xoxfhbabe 哈哈你这个说法我头回听，声音是提醒不是吓唬对吧？那你骑车会跟着发动机声控速吗

[小栗说车日常记 @kerdede]

@Huawei rest day today and I feel weirdly guilty about it

[小栗说车日常记 @kerdede]

@insidetheNFL @Lions @BarrySanders out the door by 5:40, streets were super peaceful

[Dell Technologies @Dell]

Grow your business with scalable AI, from Dell AI Factory with @NVIDIA.

[小栗说车日常记 @kerdede]

@CodewizzyX the humidity lately is a whole different enemy, respect for pushing through

[小栗说车日常记 @kerdede]

@0xDeniAi rainy runs are actually the best ones no cap

[MANTRA | The EVM L1 for RWAs @MANTRA_Chain]

Livestreaming on X and Youtube: https://t.co/oxF9kn1iDq

Send your replies in this thread or the original post to be in with a chance at the $MANTRA prize.