santosh kumar

Graduate Student @ IITM

[Mrinal @Hi_Mrinal]

by installing a local model like qwen and playing around it made me learn a lot of stuffs related to transformer, attention, inference than any youtube tutorial ever could ...

we somewhat got molded into a system that "learn first then do the thing later" thingy but actually doing the thing and making mistakes would make you learn a lot of stuffs quicker ....

[meng shao @shao__meng]

LiteLLM 把 Agent 传输层 SDK 开源了

agentchat 这个 Python SDK 让你可以用任意框架写的 AI Agent 统一接入 Slack、Teams、iMessage、WhatsApp、Discord 五个聊天平台。避免自己处理一遍 OAuth、webhook 签名、Socket 连接、去重、会话隔离这种接入工作。

架构：管道 + 协议 + 值对象
平台事件 → Channel 适配器（解码/路由）→ AgentChat 核心（去重→加会话锁→入历史→分发）
→ 应用 handler（授权、调用你的 agent）→ 回复原路送达

开源地址：https://t.co/J1NwU4SR0m

[Kirk Borne @KirkDBorne]

Informal mathematics conveys wisdom and insight beyond formal research papers >> “Structure and Randomness: Pages from Year One of a Mathematical Blog”

💥Another great book by math genius Terence Tao: https://t.co/8hcYPCUPLP 🚀

[Kirk Borne @KirkDBorne]

Download 280-page PDF math book. “Combinatorics Through Guided Discovery” at https://t.co/DdppgWQsAx

[Antonio Lupetti @antoniolupetti]

"Calculus" by Gilbert Strang is a free textbook from MIT that has become a true classic in mathematics education.

With nearly 700 pages, it provides an exceptionally clear and comprehensive introduction to calculus, covering limits, derivatives and their applications, integration techniques, the Fundamental Theorem of Calculus, differential equations, exponential and logarithmic functions, infinite series, polar coordinates, complex numbers, vectors and matrices, multivariable calculus, partial derivatives, multiple integrals, and vector calculus.

This book is an absolute must-have for anyone approaching calculus for the first time, but also an excellent reference for reviewing its fundamental concepts. I highly recommend adding it to your personal library as a valuable resource to consult whenever you need it.

https://t.co/ogv9fUj2rH

[Robert Greene @RobertGreene]

You need to be brutal with your past, so that the present and future can glisten with new, sublime possibilities.

[Shubham Mishra 🐐 @shubh6200]

MongoDB’s architecture is getting interesting.

Atlas and Infinite: a deep dive into disaggregated MongoDB, and how separating compute from storage can change the way databases scale.

A great read for anyone interested in distributed systems and database internals.

https://t.co/BT5og4cv6x

[Math Cafe @Riazi_Cafe_en]

NYU's "Deep Learning" (DS-GA 1008)
by Yann LeCun & Alfredo Canziani

Taught by Turing Award winner Yann LeCun. Covers CNNs, RNNs, attention & energy-based models.

Course Website: https://t.co/HfIXjTYJoZ
Lecture Videos: https://t.co/KyLd2eJWFh

[Puneet Patwari @system_monarch]

13 years in backend. Principal Engineer at Atlassian.

Concepts I wish I'd learned in year 2 instead of year 8:

→ p99 latency, and why averages lie
→ Connection pooling
→ Backpressure
→ Idempotency keys
→ Reading query plans (EXPLAIN ANALYZE)
→ Write amplification
→ Consistent hashing
→ Change data capture
→ The outbox pattern
→ Tail latency amplification

None of them are hard.

Every one of them would have saved me from tonnes of headaches.

[Math Cafe @Riazi_Cafe_en]

Princeton's "Distributed Systems" (COS 418)
by Michael Freedman, Kyle Jamieson & Wyatt Lloyd

Covers RPCs, logical clocks, consensus/RAFT, Spanner & more.

Course website: https://t.co/BtWNwczsVy

[Kirk Borne @KirkDBorne]

Are small language models the future? Some AI leaders say “yes”.

"How to Build and Fine‐Tune a Small Language Model: A Step-by-Step Guide for Beginners, Researchers, and Non-Programmers"

Available at https://t.co/msIjZw2arK

Build your own AI—without a PhD, expensive hardware, or industry-level resources. Whether you’re a beginner, a student, a scientist, or a domain expert, this book shows you how to create, train, fine-tune, and deploy Small Language Models (SLMs) that truly understand your field.

Most AI books explain what models are. This one teaches you to build them. You’ll go from zero to a working GPT-style model, then learn how to fine-tune, align, evaluate, and deploy it for real applications.

All chapters include ready-to-run Google Colab notebooks.

[Antonio Lupetti @antoniolupetti]

"Scaling Laws, Carefully" is an excellent article by Lilian Weng that explores the mathematical foundations of scaling laws in deep learning, explaining how model performance changes with the number of parameters, training data, and computational resources.

It covers power-law relationships, Kaplan and Chinchilla scaling laws, compute-optimal training, data limitations, and the challenges of predicting the performance of larger models.

The article includes equations, experimental results, and references to the original research papers, making it a very useful resource for understanding the mathematics behind scaling large language models.

https://t.co/1A5YOlw34V

[Suraj Sharma @suraj_sharma14]

If you learn AI Infrastructure Engineering, you will never be unemployed again.

Stage 1: Systems & GPU Foundations
- Learn: C++, Rust, CUDA memory hierarchy, PCIe vs NVLink, thread blocks and warp scheduling.

- Practice: Write a custom CUDA kernel for matrix multiplication from scratch and profile it against cuBLAS.

- Why: You cannot optimize what you do not understand at the silicon level.

Stage 2: Transformer Inference Physics
- Learn: Prefill vs Decode phases, arithmetic intensity, memory bandwidth bottlenecks, KV-cache math.

- Practice: Profile a HuggingFace model with Nsight Systems to identify the exact memory bottleneck during token generation.

- Why: LLM inference is almost always memory-bound, not compute-bound.

Stage 3: Modern Serving Engines & Batching
- Learn: vLLM, SGLang, PagedAttention, continuous batching, chunked prefill.

- Practice: Deploy a 70B model and tune chunk sizes to maximize throughput without starving decode requests during long-context prefills.

- Why: Naive batching leaves 60% of GPU VRAM wasted. PagedAttention fixes this.

Stage 4: KV-Cache & Memory Optimization
- Learn: Prefix caching, KV quantization, CPU offloading, multi-turn cache reuse.

- Practice: Build a routing proxy that directs requests with identical system prompts to the same replica to share KV blocks.

- Why: Reused cache is free speed. It can slash Time-To-First-Token (TTFT) by 80%.

Stage 5: Quantization & Compression
- Learn: FP8, INT4, AWQ, GPTQ, Sparsity, TensorRT-LLM calibration.

- Practice: Serve a model in FP8 vs FP16 and benchmark the exact perplexity drop against the latency and VRAM gains.

- Why: Quantization is the only way to fit frontier models on edge GPUs and protect your margins.

Stage 6: Kernel-Level Engineering
- Learn: Triton, FlashAttention, CUDA graphs, operator fusion.

- Practice: Write a fused Triton kernel for RMSNorm or Softmax and benchmark it against the native PyTorch implementation.

- Why: Python overhead kills inference. Fused kernels save milliseconds that compound at scale.

Stage 7: Distributed Inference & Parallelism
- Learn: Tensor parallelism, Pipeline parallelism, Expert parallelism (for MoE models).

- Practice: Shard a 405B parameter model across 8 nodes and measure the communication overhead between GPUs.

- Why: Single-GPU inference is dead for frontier models. You must master the shard.

Stage 8: Speculative Decoding
- Learn: Draft-target models, Medusa heads, acceptance rates, n-gram drafting.

- Practice: Build a pipeline where a local 3B model drafts tokens for a cloud 70B model to verify in parallel.

- Why: 2x decode speed at zero quality cost is the closest thing to a free lunch in inference.

Stage 9: Multi-Node & Hardware Interconnects
- Learn: NCCL, RDMA, InfiniBand, NVLink, Disaggregated Prefill/Decode architectures.

- Practice: Set up a multi-node cluster and profile the network latency of tensor parallelism across nodes vs within a single node.

- Why: Network latency is the new GPU bottleneck. Disaggregating prefill and decode is the 2026 meta.

Stage 10: Cluster Orchestration & GPU Scheduling
- Learn: Kubernetes GPU operators, Ray, Slurm, MIG (Multi-Instance GPU) partitioning, KEDA.

- Practice: Build a queue-based autoscaler that spins up spot GPUs based on pending inference requests and drains them when empty.

- Why: Idle H100s burn $3+/hr. FinOps and scheduling are now core infra responsibilities.

Stage 11: AI Gateways, Routing & Observability
- Learn: TTFT/ITL SLOs, semantic routing, DCGM metrics, OpenTelemetry for LLMs.

- Practice: Build a gateway that routes simple queries to a quantized local model and complex reasoning to a frontier API based on prompt complexity.

- Why: Routing protects your margins and DCGM metrics tell you when your GPUs are silently throttling.

Stage 12: Public Benchmarks & Teardowns
- Learn: Reproducible methodology, latency/throughput Pareto curves, cost-per-token analysis.

- Practice: Publish a teardown comparing vLLM vs SGLang vs TensorRT-LLM on your specific hardware with full configs.

- Why: Public proof of hardware mastery gets you hired instantly by top AI labs.

Wrappers are a commodity. AI Infrastructure is the physics of scale.

The modern AI Infra Engineer builds the silicon nervous system for global intelligence.

Bookmark and Repost!

[Soheil Feizi @FeiziSoheil]

🎥 Sharing my talk at the Simons Institute, UC Berkeley, on "Understanding the Structure of Reasoning in Language Models":

https://t.co/KLSqqrn54y

Thinking tokens are becoming a new scaling axis for AI. But does more thinking mean smarter thinking?

In this talk, I discuss four of our recent papers:

- Reasoning has structure. Reasoning traces reveal a shared "heartbeat" and recurring loops that can predict correctness. ThinkARM (ACL 2026 Oral): https://t.co/eBgzBveAfI

- Better reasoning without reward-model training. A larger model guides a smaller model's reasoning, matching a trained process reward model without reward-model training. EMNLP 2026 Main Conference: https://t.co/OVlAlIsmzT

- Knowing when to stop matters. Models can learn to stop based on their confidence, using up to 25% fewer thinking tokens at matched accuracy. ConfSFT: https://t.co/aY7B46QBGu

- Thinking harder isn't always thinking smarter. Models struggle to allocate compute across problems based on their importance, following prompts rather than payoffs. Thinking Hard, Not Smart: https://t.co/KblxVel45E

The key takeaway is that more compute doesn't guarantee better reasoning. What matters is how, when, and where models think.

Thanks to the @SimonsInstitute for hosting and to my wonderful students and collaborators!

[Math Cafe @Riazi_Cafe_en]

Stanford's "CS231n: Convolutional Neural Networks for Visual Recognition"
by Fei-Fei Li

The course that kickstarted the deep learning revolution in computer vision.
Lecture notes: https://t.co/B5NLE5Cv5k
Videos: https://t.co/hRfM4Edm6T

[Justin Skycak @justinskycak]

Abstract math is not the problem. Proofs are not the problem.

Those are not really the primary reason why math gets hard.

The biggest reason why math gets hard is:

Students are thrown into abstraction before they even understand anything concretely.

When that happens, the math just turns into symbol pushing.

Definitions, theorems, proofs, more theorems, more proofs.

But no concrete examples, no real grip on the objects.

There's nothing to hold onto, nothing to grasp, and it's backwards.

Build the concrete understanding first, make the calculations instinctive, and then the abstraction actually means something.

Further reading: https://t.co/BK2z3G3Jwe

[Mrinal @Hi_Mrinal]

there is a very well articulated conversation of founder of cockroachDB Spencer Kimball ...

https://t.co/da3yh9L5bI

[Mrinal @Hi_Mrinal]

after partially completing my rabbit hole of netflix's ai infrastructure its time to go for stripe's rabbit hole

started with how do they build their proxy and the optimization tricks to play while building one

https://t.co/oci6tOAA7L

[Addy Osmani @addyosmani]

"your edge right now is understanding the possibilities and being curious"

Using AI for software engineering is now the default. The engineers I've seen who have the edge are ambitious and have been grinding with agents for years on the job at this point. It's a compounding effect.

For teams still unsure or catching up, the smartest thing I can advise is to be even more ambitious and aggressively share what you learn with your peers. If you do all this work privately, only you benefit. If your team only gets to experiment with AI after hours, you've quietly told them using it doesn't count.

Senior engineers should be the first to make this shift. Their decades of taste in system design and failure modes is the secret sauce that makes agent output good. They should direct their experience to building with agents better safety. Then show whats working to others to replicate.

[Siddhant Mohan @siddhantmohan1]

Introducing open source AI code review. The review you pay $60 a developer for, now $0 and open source: 100+ scanners plus an AI reviewer that has to answer every finding, on your own Claude Code or Codex login, before you push.

https://t.co/O8zCUUBeAa