houseroad

[PyTorch @PyTorch]

At PyTorch Conference 2026, Lu Fang, Research Scientist, and Ilina Mitra, Engineering Manager @Meta, will present a deep dive on building high-performance recommendation inference systems.

They will detail end-to-end production workflows, covering graph capture, model splitting, and serving across large-scale platforms like Facebook and Instagram. Attendees will gain technical insights into advanced optimizations for high TPS and low latency, as well as strategies for multi-accelerator support to handle diverse machine learning workloads at scale.

Register for PyTorchCon North America today: https://t.co/jBApW8nESi

#PyTorchCon @houser0ad @ilina_mitra

[vLLM @vllm_project]

🚀 No More Train–Inference Mismatch!
We demonstrate bitwise consistent on-policy RL with TorchTitan (training) + vLLM (inference) — the first open-source run where training and inference numerics match exactly.

It only takes 3 steps:
1️⃣ Make vLLM batch-invariant (same seq → same output regardless of batching)
2️⃣ Ensure forward passes in training use identical kernels as inference
3️⃣ Add custom backward passes in PyTorch

✅ Verified on Qwen3 1.7B + GSM8K:
• batch_inv_ON (bitwise exact) → KL=0.0, faster convergence, higher reward
• batch_inv_OFF → reduced reward, instability

We audited every op, imported vLLM’s fused kernels (SiLU MLPs, RMSNorm+residual), and wrote matching backward passes. Run is fully on-policy, deterministic, and reproducible.

Next:
• Unified model code
• torch.compile support
• Perf tuning (current bitwise RL ≈2.4× slower)
• Broader model + op coverage

🔗 https://t.co/e0YChOkYm6
#vLLM #TorchTitan #RL #LLM #AIResearch