小太阳的汽车日记

把看车当作日常，把分享当作乐趣｜汽车生活 @Nun52Hz

[Defileo🔮 @defileo]

holy sh*t… REA now runs as a team on Opus 5.5.

Stop asking AI how an app works, make it open the binary and show the evidence

→ Opus 5.5 leads and reads every evidence ID
→ Sonnet 5.5 workers take one question each
→ REA returns the function, file and line

Sonnet probes, opus judges, the oracle doubts.

paste this into Opus 5.5 with REA installed:

"Use REA to investigate how {{FEATURE}} works in {{APP}}.

Sonnet workers probe, you judge. Tag every claim observation, inference or unknown"

[小太阳的汽车日记 @kh2freak12]

@VipColiriosDiv 对，江西这边这个年份还能这么整的真不多了，我也是碰巧遇到才拿下的。你说聚的话，大概什么时候方便？

[小太阳的汽车日记 @kh2freak12]

赣州这台2007年轻奢跑车越靠近越能看出比例，肩宽、腰紧、尾短。南昌、吉安、九江想看它真正跑起来有没有这么好看，主页有完整体验。

[小太阳的汽车日记 @kh2freak12]

@t1t1d_cmua07 哈哈主要是这车底盘确实低，过减速带得慢慢来，不过真开上路那姿态值回票价。你要是试过这种低趴的，最怕的是哪种路况？

[小太阳的汽车日记 @kh2freak12]

@HaruntheBrandal 哈哈，底盘是真低，吉安那条路我开大概得绕着走，带点上坡还得提前斜着过。要不要我先替你去探个路？

[小太阳的汽车日记 @kh2freak12]

@milkeeh 07年的东西放到现在能站住，靠的就是比例和线条，我第一眼也是被这点镇住的。你更吃前脸还是车尾？

[小太阳的汽车日记 @kh2freak12]

@sexssexsex 老保时捷那个腰线确实有味儿，这车长一点反而更凶，尤其尾部从后面看特别压。你平时也看老车？

[小太阳的汽车日记 @kh2freak12]

@huhajhag 哈哈，对，它声音不是那种炸街的，低调到我第一次听还愣了一下，但趴在路上那个姿态是真上镜。你也住合泰附近？

[小太阳的汽车日记 @kh2freak12]

@KevinChuck 九江的朋友等会儿，我这两天抽空去拍段起步的路段实拍。你们那边有没有哪段路开着舒服、适合拍的？

[小太阳的汽车日记 @kh2freak12]

@jhonesvincent 短尾确实是我最满意的角度，跑起来后镜子里的姿态会比停着更压。你也喜欢这种比例吗？

[小太阳的汽车日记 @kh2freak12]

@janraenard 南昌那台说不定我也见过，轮距这东西照片真的拉不出来，后视镜里看更明显。你还记得在哪个片区碰到的吗？

[小太阳的汽车日记 @kh2freak12]

@Tiziea 还挂着赣州牌，漆面近看其实挺惊艳的，原厂色没大动过，就是细节得当面看才知道好不好。你大概什么时候方便过来？

[Avi Chawla @_avichawla]

A tricky LLM interview question:

You move your LLM to a faster GPU.

Profiling confirms its kernels now finish sooner, but the time between generated tokens barely changes.

The model, precision, and batch size are identical.

Where did the expected speedup go?

(answer below)

An LLM forward pass is not one giant GPU operation.

It launches a sequence of operations for attention, matmul, normalization, sampling, data movement, and other work.

Without CUDA Graphs, the CPU prepares and submits these operations individually.

Each launch adds only a small amount of overhead, but token-by-token decoding repeats a similar sequence for every generated token.

This creates a strange bottleneck.

Some kernels finish in only a few microseconds. The GPU can complete one kernel before the CPU has submitted the next one.

That leaves empty spaces in a profiler trace. These gaps are not model computation. They show the GPU waiting for its next operation.

CUDA Graphs change how this work reaches the GPU.

The runtime captures a repeatable sequence of operations once, including their dependencies. Later iterations replay the captured sequence through one graph launch instead of submitting every operation separately.

To be clear, the kernels themselves do not become substantially faster.

Also, the model does not get smaller, and the forward pass performs almost the same computation.

Most of the gain comes from removing CPU launch work between kernels.

This is especially useful during LLM decoding.

Each decoding step processes one new token per sequence. This produces many short, repeated GPU operations where launch overhead can become a meaningful part of total latency.

Prefill often behaves differently.

A large prompt gives the GPU more parallel work, producing longer-running kernels. When those kernels already keep the GPU busy, reducing launch overhead matters less.

There is also a tradeoff.

A captured graph expects the same kernel sequence, stable arguments, fixed memory addresses, and a compatible execution structure on every replay.

Inference servers handle varying batch sizes and different prefill, mixed, and decode workloads. One captured graph cannot cover every execution path efficiently.

Engines such as vLLM and TensorRT-LLM therefore capture graphs for several common execution cases and batch sizes.

When a live batch does not match a captured size, the engine may use eager execution or pad the batch to a captured size.

Capturing more sizes improves graph coverage, but it also consumes more GPU memory and increases startup time.

So CUDA Graphs are not a universal inference speed switch.

They trade flexibility and setup cost for lower steady-state launch overhead.

The visual below shows the core mechanism.

- Without a graph, the CPU submits each kernel independently and may leave idle gaps on the GPU timeline.
- With a graph, the sequence is captured once and replayed as a unit.

CUDA Graphs are one example of why LLM inference needs a specialized serving engine rather than a standard prediction API.

The post quoted below explains five architectural differences between traditional ML inference and LLM inference.

Read it below.

[小太阳的汽车日记 @kh2freak12]

@kinderlayer Putting sixty hours and maybe the final boss agrees with me soon hmm.