Wuttichai Buraso

Senior Software Engineer

[Yingcheng Charles Wu @Charles_Y_Wu]

Excited to introduce RSI-Anything!

One RSI model that works across coding, GUI, planning, or even Science tasks.

Let's RSI Everything!

📄 Paper: https://t.co/HOD4782oWP
💻 Code: https://t.co/ttGFHVmfdt

#RSI #WorldModels #RSI #AIAgents #AIforScience #JEPA

[freeCodeCamp.org @freeCodeCamp]

Building a custom chatbot can be tricky, especially if you want to keep the info private & efficient.

So here, @atuo_elabonga teaches you how to create a local Retrieval-Augmented Generation application using Ollama & ChromaDB in R.

By the end, you'll have a Shiny interface for your chatbot that fetches relevant info while staying private.

https://t.co/w95H9sYioD

[Pallavi @pallavishekhar_]

AI Engineering Interview Questions and Answers

https://t.co/Ms8Q6M9N71

[Ding @dingyi]

我愿称之为史上最强 skills

https://t.co/PKMyj1Qd1s

[PyTorch @PyTorch]

Running large Mixture-of-Experts models across pods creates two major challenges: expert dispatch and combine traffic, and moving the KV cache between prefill and decode.

At #PyTorchCon North America 2026, Ravi Gupta, Rishi Madduri, and Shiksha Patel (@AMD) will present “MoRI + vLLM: Wide Expert Parallelism and RDMA KV-Cache Transfer for Disaggregated MoE Serving on AMD.”

The poster will show how bringing the MoRI communication stack into @vllm_project enables Wide Expert Parallelism across pods using RDMA and moves the KV cache for disaggregated prefill and decode on AMD Instinct GPUs.

They will also cover correctness and stability issues encountered when spanning pods and share vLLM benchmark results for long, high-concurrency workloads.

Register for PyTorch Conference North America 2026: https://t.co/jBApW8nESi

[Suraj Sharma @suraj_sharma14]

If you learn AI Infrastructure Engineering, you will never be unemployed again.

Stage 1: Systems & GPU Foundations
- Learn: C++, Rust, CUDA memory hierarchy, PCIe vs NVLink, thread blocks and warp scheduling.

- Practice: Write a custom CUDA kernel for matrix multiplication from scratch and profile it against cuBLAS.

- Why: You cannot optimize what you do not understand at the silicon level.

Stage 2: Transformer Inference Physics
- Learn: Prefill vs Decode phases, arithmetic intensity, memory bandwidth bottlenecks, KV-cache math.

- Practice: Profile a HuggingFace model with Nsight Systems to identify the exact memory bottleneck during token generation.

- Why: LLM inference is almost always memory-bound, not compute-bound.

Stage 3: Modern Serving Engines & Batching
- Learn: vLLM, SGLang, PagedAttention, continuous batching, chunked prefill.

- Practice: Deploy a 70B model and tune chunk sizes to maximize throughput without starving decode requests during long-context prefills.

- Why: Naive batching leaves 60% of GPU VRAM wasted. PagedAttention fixes this.

Stage 4: KV-Cache & Memory Optimization
- Learn: Prefix caching, KV quantization, CPU offloading, multi-turn cache reuse.

- Practice: Build a routing proxy that directs requests with identical system prompts to the same replica to share KV blocks.

- Why: Reused cache is free speed. It can slash Time-To-First-Token (TTFT) by 80%.

Stage 5: Quantization & Compression
- Learn: FP8, INT4, AWQ, GPTQ, Sparsity, TensorRT-LLM calibration.

- Practice: Serve a model in FP8 vs FP16 and benchmark the exact perplexity drop against the latency and VRAM gains.

- Why: Quantization is the only way to fit frontier models on edge GPUs and protect your margins.

Stage 6: Kernel-Level Engineering
- Learn: Triton, FlashAttention, CUDA graphs, operator fusion.

- Practice: Write a fused Triton kernel for RMSNorm or Softmax and benchmark it against the native PyTorch implementation.

- Why: Python overhead kills inference. Fused kernels save milliseconds that compound at scale.

Stage 7: Distributed Inference & Parallelism
- Learn: Tensor parallelism, Pipeline parallelism, Expert parallelism (for MoE models).

- Practice: Shard a 405B parameter model across 8 nodes and measure the communication overhead between GPUs.

- Why: Single-GPU inference is dead for frontier models. You must master the shard.

Stage 8: Speculative Decoding
- Learn: Draft-target models, Medusa heads, acceptance rates, n-gram drafting.

- Practice: Build a pipeline where a local 3B model drafts tokens for a cloud 70B model to verify in parallel.

- Why: 2x decode speed at zero quality cost is the closest thing to a free lunch in inference.

Stage 9: Multi-Node & Hardware Interconnects
- Learn: NCCL, RDMA, InfiniBand, NVLink, Disaggregated Prefill/Decode architectures.

- Practice: Set up a multi-node cluster and profile the network latency of tensor parallelism across nodes vs within a single node.

- Why: Network latency is the new GPU bottleneck. Disaggregating prefill and decode is the 2026 meta.

Stage 10: Cluster Orchestration & GPU Scheduling
- Learn: Kubernetes GPU operators, Ray, Slurm, MIG (Multi-Instance GPU) partitioning, KEDA.

- Practice: Build a queue-based autoscaler that spins up spot GPUs based on pending inference requests and drains them when empty.

- Why: Idle H100s burn $3+/hr. FinOps and scheduling are now core infra responsibilities.

Stage 11: AI Gateways, Routing & Observability
- Learn: TTFT/ITL SLOs, semantic routing, DCGM metrics, OpenTelemetry for LLMs.

- Practice: Build a gateway that routes simple queries to a quantized local model and complex reasoning to a frontier API based on prompt complexity.

- Why: Routing protects your margins and DCGM metrics tell you when your GPUs are silently throttling.

Stage 12: Public Benchmarks & Teardowns
- Learn: Reproducible methodology, latency/throughput Pareto curves, cost-per-token analysis.

- Practice: Publish a teardown comparing vLLM vs SGLang vs TensorRT-LLM on your specific hardware with full configs.

- Why: Public proof of hardware mastery gets you hired instantly by top AI labs.

Wrappers are a commodity. AI Infrastructure is the physics of scale.

The modern AI Infra Engineer builds the silicon nervous system for global intelligence.

Bookmark and Repost!

[freeCodeCamp.org @freeCodeCamp]

As your team adds more AI features, LLM calls can become harder to manage.

In this guide, @t_koded explains how the LLM Gateway Pattern centralizes model traffic in Kubernetes.

You'll learn how to manage API keys, routing, rate limits, caching, failover, token usage, and observability in one place.

https://t.co/EGU0yLM3Tn

[Tianyu Fu @fuvty123]

🔥 TokenRouter is out! Efficient token-level LLM routing, now ready for serving.

🚀 2.01–64.15× higher throughput vs. serving baselines
✍️ Just 3 functions: send, receive, route

Bring your routing algorithm. We handle the serving.
https://t.co/NApSAgw9Vg

[Tech with Mak @techNmak]

You can literally type a sentence and watch a Transformer process it.

This interactive explainer runs a GPT-style model in the browser and lets you follow the whole path:

tokenization → embeddings → Q/K/V → attention → MLP → logits → next-token probabilities

You can inspect attention maps, intermediate values, and change temperature, top-k and top-p to see how generation changes.

If Transformers still feel like boxes connected by arrows, this is worth opening.

https://t.co/vkyxOqDVll

[Aritra 🤗 @ariG23498]

People looking for blog posts to learn about profiling in pytorch

1: https://t.co/PtdqmpeicD
2: https://t.co/bYh1Px24gL
3: https://t.co/rCzWVOqYbC

Happy weekend! 😎

[Pallavi @pallavishekhar_]

Learn LLM internals step by step - from tokenization to attention to inference optimization.

- Tokenization
- Byte Pair Encoding
- Math behind Attention - Q, K, and V
- Math behind √dₖ Scaling Factor in Attention
- Causal Masking in Attention
- Math Behind Backpropagation
- Math Behind Cross-Entropy Loss
- Decoding Transformer Architecture
- Feed-Forward Networks
- Batch Normalization vs Layer Normalization
- KV Cache
- Paged Attention
- Flash Attention
- Speculative Decoding
- Continuous Batching
- Prompt Caching
- Prefill vs Decode: LLM Inference Optimization
- Mixture of Experts Explained
- Grouped Query Attention
- Math Behind RoPE (Rotary Position Embedding)
- RMSNorm (Root Mean Square Layer Normalization)
- LoRA: Low-Rank Adaptation
- Large Reasoning Models (LRMs)
- Reinforcement Learning from Human Feedback (RLHF)
- Harness Engineering

Learn here: https://t.co/z7O1NVjMwq

[Leonard Rodman @RodmanAi]

10 GITHUB REPOS THAT GIVE AI AGENTS A MEMORY

Most AI agents can answer questions.

But can they remember what happened yesterday?

These 10 open-source projects are exploring how to give AI persistent memory:

1. Mem0 — Remember user preferences and past conversations.
https://t.co/nwgygvAyoF

2. Zep — Retrieve useful context from previous interactions.
https://t.co/fL5f3iBR8L

3. Graphiti — Build temporal knowledge graphs that track changing facts.
https://t.co/QHZXpJhqdV

4. Memary — Explore memory infrastructure for AI agents.
https://t.co/Z43eY1WAR7

5. Cognee — Turn raw data into connected knowledge for AI.
https://t.co/9bFdFGmbdQ

6. Letta — Build stateful agents that manage memory across sessions.
https://t.co/CdzwQtON5K

7. Memvid — Store and retrieve information with portable, file-based memory.
https://t.co/ujcNu5pdOk

8. Supermemory — Build memory systems that retrieve relevant context when needed.
https://t.co/CQeb0OPmcG

9. SimpleMem — Explore efficient long-term memory for LLM agents.
https://t.co/Je0YDxwpAy

10. Engram — Explore persistent memory for AI workflows.
https://t.co/tHOqaJBu8s

Save this list.

The next big leap in AI agents might not be a smarter model.

It might be an agent that actually remembers.

[David Hendrickson @TeksEdge]

New Small Local Model! JetBrains squeezed a 12B coding model into an 8.1GB Q4. I have to test against Qwen3.5-9B.

… w/only 2.5B parameters active per token.

Mellum2.1-12B-A2.5B was post-trained to work inside repositories and do things like …
🔎 explore the codebase
✏️ edit files
🧪 work with shell/tests
✅ check its own changes

JetBrains says it ran millions of sandboxed RL runs across thousands of environments during post-training.

And official local quants are already up!
🧠 12B total parameters
⚡ 2.5B active/token
💾 Q4_K_M → 8.1GB
💾 MXFP4_MOE → 7.0GB
📚 131K context
🔓 Apache 2.0
🦙 llama.cpp
🦙 Ollama
🖥️ LM Studio

JetBrains reports for Mellum2.1 Thinking
💻 SWE-bench Verified → 47%
⌨️ LiveCodeBench v6 → 82%
🧮 AIME 25/26 → 83.3%

Stats
12B model
→ 2.5B active
→ 8.1GB Q4
→ local coding agent

🔗 Link in ALT.

[Jiaxuan Zou @SmartPig_Joe]

1/5
New blog: How to Build a Scientific Scaling Ladder.

I've spent the past three months studying pretraining and scaling. This guide brings together public research on using small runs to predict large-model training configurations and performance.

https://t.co/KJ7qL04GMF

[Shubham Mishra 🐐 @shubh6200]

This is all you need for your next System Design interview.

Bookmark these 👇

https://t.co/Y2HIJOrUDo

https://t.co/TJLE9kftMc

[Uber Engineering @UberEng]

https://t.co/hZjJWYlnye

[Karthik @kartikktwt]

If you're writing go for your projects or work. I'd highly suggest reading about go's GC and scheduler if you've not before

those are work of art. literally.
and you'll understand why ppl prefer GO

i've written about GO's GC in my words before
give it a read if you have time

https://t.co/DjPHzD78yo

[leonie @helloiamleonie]

this is a great learning resource for anyone getting into decision models.

just played around with this with LFM2.5-1.2B-Instruct and had a working decision model after 100s on an A100.

[Gauri Tripathi @GauriTripa94282]

JEV vs LLM — they solve different problems.

JEV evaluates predefined decisions with typed answers and confidence.

LLMs generate text, code, summaries, and explanations.

① JEV: Choose, score, or verify
② LLM: Generate, reason, and explain

The key is knowing when to use each.

[Rohan Paul @rohanpaul_ai]

New Microsoft paper shows that agents can learn software skills from raw usage logs by keeping only skill edits that better predict users' next actions.

i.e. You do not need a live test environment to check whether a new agent skill helps, because next-action accuracy on old logs tracked live success.

Usage logs hold lots of know-how, but they record no goals, often mix several tasks, and cannot be replayed. Earlier methods, like Agent Workflow Memory, need goal-labeled examples or a live environment to test changes.

TeleTune guesses each session's goal and has the model predict every logged action using a text skill library. Wrong guesses suggest library edits, and an edit stays only if accuracy rises on held-out logs.

If your product records user activity, mine it for agent skills and judge each change by next-action accuracy on held-out logs.

– arxiv. org/abs/2610.05437

Title: "TeleTune: Evolving Agent Skills From Offline Telemetry"