Vensen

Pursuing operator-level train-inference consistency and batch invariance.
@RLKernel Author. AI Infra Engineer.

[RL-Align @RLKernel]

Thanks to @qingke_ai for having us.
RL-Kernel core team member Jiajie Li will share how we tackle logP discrepancies and achieve train-inference consistency in LLM RL post-training.

[Vensen @vensen6521]

Happy to be a PyTorch Foundation Ambassador for 2026–2028!

Thanks to everyone building alongside us. I’ll keep working on @RLKernel to make training and rollout bitwise consistent while keeping performance high.

Still learning, still building. #PyTorch

https://t.co/KvXqOK9Mre

[vLLM @vllm_project]

Training and rollout logprobs matched bit for bit on ROCm. The @RLKernel team integrated RL-Align/RL-Kernel with vllm-project/vime. A 200-step Qwen3-8B GRPO run on 8× @AMD MI300X recorded zero logprob mismatches between Megatron training and vLLM rollout.

The strict path aligns reduction order, intermediate precision, rounding points, and math primitives across both sides.

Deep dive:
https://t.co/FVJHoyoo3t

[RL-Align @RLKernel]

RL-Kernel v0.1.0 is now available. 🎉

Bitwise train–inference consistency across parallel configurations and heterogeneous execution engines for RL post-training.

With Qwen3-8B on a full vime pipeline, RL-Kernel recorded zero train–rollout LogP mismatches across all 200 training steps.

1,129 commits from 29 contributors.

Highlights:

🎯 Strict numerical alignment across rollout and training engines
⚡ 68.7% higher rollout throughput and 8.2% lower end-to-end step time on NVIDIA H100
🔶 AMD Instinct MI300X: zero mismatches across 9,400,614 active-token LogP elements from 1,600 samples
🔁 End-to-end integration with vime, vLLM rollout, and Megatron-LM training
🧩 Deterministic Attention, dense FFN, LogP, reduction, and collective operators
🌐 NVIDIA CUDA and AMD ROCm support, with initial Ascend integration and MUSA adaptation underway

GitHub 👇
https://t.co/UThkQhmeCC

[RL-Align @RLKernel]

Inference vLLM and training Megatron execution graphs never quite align. Single-operator boundaries break down, and treating kernels from one side as a drop-in contract simply doesn't work. That's why we built RL-Kernel 🚀

Today, we’re dropping our first milestone: an operator-level training-inference consistency analysis & roadmap preview for DeepSeek V4 Flash MoE. Our goal is to build reusable, hardware-agnostic infrastructure bridging both worlds.

If your daily stack involves writing low-level CUDA/Triton kernels or wrestling with vLLM & Megatron MoE distributed comms, you'll feel right at home here. We're fully open-source and looking for core contributors.

We'll be dropping the DS V4 roadmap in our GitHub issues over the next week or so, complete with detailed task breakdowns. External contributors are more than welcome to jump in! Grab an issue, join us, and let's ship this together!

👇 Read the Blog :
https://t.co/DqkTYzTZOa

#RLKernel #DeepSeek #AIInfrastructure #RLHF #CUDA #Triton #MoE #RLAlign #ROCm

[Vensen @vensen6521]

@sheriyuo To clarify, this is the QR code for the new community WeChat group. Thank you.

[Vensen @vensen6521]

@Islander148182 @sheriyuo Hey, You need to register a WeChat account, and then I will add you on WeChat.

[Vensen @vensen6521]

@751Kim @sheriyuo Hey bro, I've added you on WeChat.

[Vensen @vensen6521]

Thank you so much for the support, Xiuyu @sheriyuo! We are on a mission to achieve bitwise operator-level train-inference consistency across heterogeneous engines, unlocking extreme memory efficiency for GRPO, PPO, and beyond. Really appreciate you sharing this with the community! 🚀

[Vensen @vensen6521]

Inspired by Horace He's @thinkymachines post on LLM Nondeterminism, we are building RL-Kernel for scalable post-training.

Single-GPU batch-invariant ops are nearly locked. Now tackling the real beast: cross-TP determinism (vLLM TP>1 vs FSDP TP=1) & the memory wall.

Would love your harsh feedback on our roadmap! https://t.co/f9D4GQOFIE

[青稞社区 @qingke_ai]

https://t.co/XQlEDd38eJ

[Xiuyu Li @sheriyuo]

The RL-Kernel project has just been open sourced on GitHub.

RL-Kernel is a low level kernel library designed for RLHF training of LLMs, with a particular focus on algorithms such as GRPO and PPO. It is deeply integrated with FlashInfer, with future plans to support CUTLASS and FlyDSL. The project includes custom implementations of sampling, prefix shared attention, TMA, and other performance critical kernels.

According to the authors, some high frequency training components achieve up to 163× speedups while also reducing memory usage.

The team is currently looking for collaborators with access to multi GPU clusters such as A100, H100, or MI300X systems. AI labs, infra teams, and cloud providers interested in testing large scale tensor parallel performance are welcome to get in touch.

GitHub: https://t.co/kotKsLkR6m
Zhihu: https://t.co/IkQWUuEBEN

They would also appreciate issues, discussions, benchmark results, and kernel contributions from the community.

[Vensen @vensen6521]

@suzatweet To push frameworks like slime to their absolute limits, we open-sourced RL-Kernel: a micro-level operator library that plugs into Megatron and sglang to enforce strict train-inference consistency and break the RL memory wall. https://t.co/AqJ53FSwPB

[Vensen @vensen6521]

1. How to prove and implement operator-level consistency?Tying into our ongoing WS1 sprint this month, we discussed how to guarantee that the same sequence strictly produces identically aligned logprobs, whether batch=1 or batch=N, and regardless of padding configurations. Key discussion points include:
1-1: Establishing ground-truth validation benchmarks and numerical contracts.
1-2: Implementation strategies for batch-invariant matmul / GEMM (the area with the highest technical risk).
1-3: Consistency auditing for core components like RMSNorm, Attention, and RoPE.

2. Technical roadmap: From single-GPU forward passes to distributed multi-GPU parallelismReal-world industrial training always starts with multiple GPUs; single-GPU autonomy is just the first step. In the WS2 phase, we are shifting our focus to production scenarios:
2-1: How to achieve TP-invariant reductions (ensuring identical execution order whether TP=1 or TP>1) and SP-aware logprob reductions.
2-2: The hardest cross-configuration alignment: handling scenarios where Rollout and Training use entirely different parallel strategies.
2-3: Cross-platform numerical alignment experiences across hardware backends like CUDA, Triton, and ROCm.

3. Two open design discussions (RFC)
3-1: The batch-invariant operator suite.
3-2: Proposals for a non-intrusive integration with vLLM rollouts.

While teams like Thinking Machines, sglang, and DeepSeek-V4 have explored batch-invariance on the inference side, our goal is to bridge this gap across the entire RL training-inference pipeline and deliver it as a reproducible, open-source operator library.

If you work on low-level kernel optimization or have been plagued by RL training-inference inconsistencies, we highly recommend checking out the replay.

Livestream Time:
Last Saturday (June 20th), 7:00 PM - 9:00 PM
Roadmap: https://t.co/f9D4GQPdyc

[Vensen @vensen6521]

RL-Kernel's First Public Livestream: Achieving Operator-Level Training-Inference Consistency

Anyone who has worked on Reinforcement Learning (RL) post-training has likely stumbled into a hidden pitfall: given the exact same model weights and inputs, the logprobs computed by the rollout engine (vLLM / sglang) and the training engine (Megatron / FSDP) simply don't match. This discrepancy leads to KL drift and contaminates policy updates.

The root cause lies at the operator level. For the exact same reduction operation, changes in batch size, parallel configurations, or padding layouts can alter the order of floating-point accumulation, resulting in numerical drift. While framework-level alignments (like vime) can mitigate this, the drift cannot be completely eradicated unless the underlying operators are inherently batch-invariant.

We are working on a solution to ensure that RL post-training operators yield absolutely consistent logprobs across different batches, parallel setups, and padding layouts. This is exactly what the open-source project RL-Kernel aims to solve: training-inference consistency at the operator level.

Last Saturday, we hosted a livestream to walk through our roadmap for the second half of 2026, breaking it down into three core areas:

[Vensen @vensen6521]

III. Open Design RFC
We're building a reproducible batch-invariant operator suite & non-intrusive vLLM integration.

If you're into low-level optimization, CUDA/Triton, or face VRAM/throughput issues in RL, join us!

Roadmap: https://t.co/XHti0fcuW3

4/4

[Vensen @vensen6521]

Ever hit the hidden pitfall in RL post-training?

Same weights & inputs, but rollout (vLLM) & training (Megatron) logprobs mismatch, causing KL drift.

Introducing RL-Kernel: Open-source operator-level training-inference consistency.

Repo: https://t.co/AqJ53FSwPB

1/4

[Vensen @vensen6521]

II. Distributed Multi-GPU (WS2)
Single-GPU self-consistency is just step one for production. Next up:
1. TP-invariant & SP-aware logprob reduction
2. Cross-config alignment (when rollout & training parallel strategies differ)
3. Cross-platform: CUDA, Triton, ROCm

3/4

[Vensen @vensen6521]

@liulicheng10 use this post and make a brief introduce our RL-Kernel project, this project main focus on resolve RL scenario problems by use CUDA and Triton kernels. details see https://t.co/KHcxRGGAsO, Thank you everyone.