Patricio Perez

Mexico

[wafer @wafer_ai]

we launched the most comprehensive ai performance engineering repo in the world

and we're posting every single resource

follow and save to keep up with the series. links in thread 🧵

part 10: CUDA Programming Guide

making an LLM faster means understanding where execution is waiting. an attention kernel can wait for its next tile of data, while the serving runtime can leave the GPU waiting for the next launch.

NVIDIA's CUDA Programming Guide explains the mechanisms behind those waits. for a modern AI performance engineer working in PyTorch, Triton, or CUDA, it helps with:

- tuning attention, matmul, and normalization kernels. the SIMT and CUDA Tile chapters connect layouts and data reuse to GPU execution. the resource guidance explains why larger tiles or more fusion can introduce register spills.

- overlapping data movement with computation. asynchronous copies, TMA, and producer/consumer pipelines explain how supported GPUs can load the next tile while processing the current one, with the synchronization required to reuse buffers safely.

- managing growing KV caches. the virtual memory chapter explicitly connects its allocation model to LLM serving: reserve an address range, then map additional physical memory into it as the cache grows, without relocating existing data.

- reducing overhead between inference kernels. CUDA Graphs reduce repeated CPU launch setup, while stream-ordered allocation avoids broad synchronization around temporary memory. these matter when launch and allocation costs contribute to request latency.

- diagnosing stalls across GPUs. the guide explains peer access and communication dependencies, including how a compute kernel's completion fence can wait on unrelated NCCL writes. memory synchronization domains can reduce that interference on supported hardware.

- evaluating CPU memory offload. the memory chapters explain how residency, migration, and interconnect traffic affect access cost. that helps assess the latency and capacity tradeoffs of keeping model data in CPU memory on supported systems.

- validating an optimization before deploying it. the appendices explain how reduction order, accumulation precision, and fast math can change results, and which hardware capabilities and resource limits a kernel depends on.

when a training step or inference request slows down, these details help narrow the next experiment: change the kernel, move less data, adjust allocation, or remove an unnecessary wait.

[Patricio Perez @Pato_perez02]

Vamos cabezas de queso #foximpactonfl @FOXImpactoNFL @FOXSportsMX #gopackgo #jordanlove

[Elon Musk @elonmusk]

He is a very promising candidate

[Elon Musk @elonmusk]

Every parent should watch this

[TE⊕F⊕R⊕S @TEOFOROS]

Muy interesante lo que dice el Papa Francisco en este extracto, no se lo pierdan!

[Emmanuel Rincón @EmmaRincon]

Trump: "¿Sabes lo que significa woke? Significa que eres un perdedor... Todo lo woke se convierte en una mierda".

[P. Gabriel María Abascal LC ✭ @pgabascal]

Compórtate como hombre:

Levántate temprano
Haz oración
Cumple la Voluntad de Dios
Acude a los sacramentos
Trabaja duro
Cuida a los tuyos
Lee
Rodéate de buenos amigos
No te quejes
Haz ejercicio
Forma a otros hombres
Vive el presente
Invoca a la Virgen María

[Emmanuel Rincón @EmmaRincon]

🇺🇸 | Trump: "Vamos a derrotar el culto a la ideología de género y reafirmar que Dios creó dos géneros llamados hombre y mujer. No vamos a permitir que los hombres jueguen en los deportes femeninos... vamos a salvar la dignidad de las mujeres...".

[VSports Team @VSportsTM]

Ney: "¿Dónde está el campeón del mundo?".

[Sudanalytics @sudanalytics_]

Todo esto en 18 meses. 🤯🇦🇷

[ESPN FC @ESPNFC]

Lionel Messi really took the 'no international trophies' comments personally 🍵

[Iniestazo @INIE8TAZO]

Grabad este vídeo en vuestra retina. La mayor leyenda que veréis jamás.

El dueño del futbol mundial.

[Barça Universal @BarcaUniversal]

Iconic.

[NewsDigitales @NewsDigitalesAr]

La mejor discusión que escuché 😂

[VSports Team @VSportsTM]

Nunca hay que dejar de intentar.

[Martin Liberman @libermanmartin]

Un número 1 que estuvo rodeado de malas personas. Tipos que se creyeron a su altura y no pueden atarle los cordones( Bruno, Bernardo, etc)
Un entrenador traidor que se creyó el creador de la Coca Cola.
Miserables, llenos de envidia!

https://t.co/yeJupMQuxO

[DjMaRiiO @DjMaRiiO]

QUÉ MIRÁS BOBO
ANDÁ PA ALLÁ BOBO