RL-Align

High-performance RL post-training infra. Achieves bitwise train-inference consistency across heterogeneous engines & extreme memory efficiency for GRPO, PPO.

[RL-Align @RLKernel]

@ok1mraise @vllm_project @AMD Thanks! We'd love to hear from others trying to reproduce the results. Feel free to share your findings or any issues on GitHub.

[RL-Align @RLKernel]

Zero mismatches. Bit-for-bit aligned. 🔥
Incredibly proud of our team's work integrating with vime @vllm_project. Rock-solid RL pipelines are now a reality on @AMD ROCm.

Read the deep dive below and star our repo⭐ https://t.co/0euenP62l5

[vLLM @vllm_project]

Training and rollout logprobs matched bit for bit on ROCm. The @RLKernel team integrated RL-Align/RL-Kernel with vllm-project/vime. A 200-step Qwen3-8B GRPO run on 8× @AMD MI300X recorded zero logprob mismatches between Megatron training and vLLM rollout.

The strict path aligns reduction order, intermediate precision, rounding points, and math primitives across both sides.

Deep dive:
https://t.co/FVJHoyoo3t

[RL-Align @RLKernel]

@SophiaAiExpert @vllm_project @AMD Thank you! Getting the full path aligned, from kernels to collectives, took a lot of careful debugging. Appreciate the encouragement!

[RL-Align @RLKernel]

@dstevens33700 @vllm_project @AMD Thanks for sharing that. Making these mismatches easier to isolate is a big part of why we're building RL-Kernel. We'd be interested to hear more about the cases you ran into.

[RL-Align @RLKernel]

@MiraSynth0 @vllm_project @AMD Thanks! A lot of work went into this across RL-Kernel, vime, and AMD. Really appreciate the support!

[RL-Align @RLKernel]

@CKPillaiAI @vllm_project @AMD Agreed, thanks for the suggestion. A small train and rollout parity test in CI would help catch numerical regressions after kernel or compiler upgrades, before committing to a long training run.

[RL-Align @RLKernel]

Thanks to @qingke_ai for having us.
RL-Kernel core team member Jiajie Li will share how we tackle logP discrepancies and achieve train-inference consistency in LLM RL post-training.

[青稞社区 @qingke_ai]

大模型 RL 后训练中，Rollout 与 Training 往往由不同引擎执行。即使模型、权重和 token 完全相同，仍可能因浮点归约顺序、并行分片、通信合并树及精度转换边界不同，产生 LogP 偏差，进而影响 PPO/GRPO 的 importance ratio、clipping 与训练稳定性。

北京时间 10月20日（周二）晚8点，#青稞社区 邀请到 RL-Kernel 核心 Member、西安交通大学硕士研究生李佳杰，以 @RLKernel RL-Kernel v0.1.0 为例，介绍训推差异的定位方法、关键算子与数值契约的对齐方式

以及固定 replay、逐层比对和端到端 200-step 验证流程，并展示 Qwen3-8B 在 CUDA 与 ROCm 环境下的验证结果。最后讨论 PD 跨机分离与异步 RL 场景的一致性边界和后续接入计划。

[青稞社区 @qingke_ai]

大模型 RL 后训练中，Rollout 与 Training 往往由不同引擎执行。即使模型、权重和 token 完全相同，仍可能因浮点归约顺序、并行分片、通信合并树及精度转换边界不同，产生 LogP 偏差，进而影响 PPO/GRPO 的 importance ratio、clipping 与训练稳定性。

北京时间 10月20日（周二）晚8点，#青稞社区 邀请到 RL-Kernel 核心 Member、西安交通大学硕士研究生李佳杰，以 @RLKernel RL-Kernel v0.1.0 为例，介绍训推差异的定位方法、关键算子与数值契约的对齐方式

以及固定 replay、逐层比对和端到端 200-step 验证流程，并展示 Qwen3-8B 在 CUDA 与 ROCm 环境下的验证结果。最后讨论 PD 跨机分离与异步 RL 场景的一致性边界和后续接入计划。

[RL-Align @RLKernel]

Big congratulations to our core maintainer @vensen6521 , on being selected as a PyTorch Foundation Ambassador (2026–2028).

We’re grateful to the community for its support, and to everyone contributing to RL-Align. We’ll keep learning, working together, and building open, reliable AI infrastructure, one step at a time.

There’s plenty more to build.

#PyTorch #RLAlign #OpenSource #AIInfrastructure

[Vensen @vensen6521]

Happy to be a PyTorch Foundation Ambassador for 2026–2028!

Thanks to everyone building alongside us. I’ll keep working on @RLKernel to make training and rollout bitwise consistent while keeping performance high.

Still learning, still building. #PyTorch

https://t.co/KvXqOK9Mre

[RL-Align @RLKernel]

Every mismatch facing the unknown deserves a deterministic response.
Who says open source only lives in terminals?

We want to build a warm community.

To thank every Contributor, Member, and Partner for building RL-Kernel with us, we created custom textured postcards.

Printed on the front:

Every Rollout into the unknown deserves a deterministic response.

These postcards won't be sold. The first 50 Contributors, Members, and Partners will get one mailed directly from Singapore.

Code is logical, but the people building an open-source project from scratch are real and warm.

When this postcard lands on your desk, remember: every PR you submitted is a real part of RL-Align (https://t.co/t6QDMvxmo1).

See you in the next rollout. ❤️

#RLAlign #RLKernel #RL #AIInfra #Github

[RL-Align @RLKernel]

RL-Kernel v0.1.0 is now available. 🎉

Bitwise train–inference consistency across parallel configurations and heterogeneous execution engines for RL post-training.

With Qwen3-8B on a full vime pipeline, RL-Kernel recorded zero train–rollout LogP mismatches across all 200 training steps.

1,129 commits from 29 contributors.

Highlights:

🎯 Strict numerical alignment across rollout and training engines
⚡ 68.7% higher rollout throughput and 8.2% lower end-to-end step time on NVIDIA H100
🔶 AMD Instinct MI300X: zero mismatches across 9,400,614 active-token LogP elements from 1,600 samples
🔁 End-to-end integration with vime, vLLM rollout, and Megatron-LM training
🧩 Deterministic Attention, dense FFN, LogP, reduction, and collective operators
🌐 NVIDIA CUDA and AMD ROCm support, with initial Ascend integration and MUSA adaptation underway

GitHub 👇
https://t.co/UThkQhmeCC

[RL-Align @RLKernel]

RL-Kernel v0.1.0 is now available. 🎉

Bitwise train–inference consistency across parallel configurations and heterogeneous execution engines for RL post-training.

With Qwen3-8B on a full vime pipeline, RL-Kernel recorded zero train–rollout LogP mismatches across all 200 training steps.

1,129 commits from 29 contributors.

Highlights:

🎯 Strict numerical alignment across rollout and training engines
⚡ 68.7% higher rollout throughput and 8.2% lower end-to-end step time on NVIDIA H100
🔶 AMD Instinct MI300X: zero mismatches across 9,400,614 active-token LogP elements from 1,600 samples
🔁 End-to-end integration with vime, vLLM rollout, and Megatron-LM training
🧩 Deterministic Attention, dense FFN, LogP, reduction, and collective operators
🌐 NVIDIA CUDA and AMD ROCm support, with initial Ascend integration and MUSA adaptation underway

GitHub 👇
https://t.co/UThkQhmeCC

[RL-Align @RLKernel]

Inference vLLM and training Megatron execution graphs never quite align. Single-operator boundaries break down, and treating kernels from one side as a drop-in contract simply doesn't work. That's why we built RL-Kernel 🚀

Today, we’re dropping our first milestone: an operator-level training-inference consistency analysis & roadmap preview for DeepSeek V4 Flash MoE. Our goal is to build reusable, hardware-agnostic infrastructure bridging both worlds.

If your daily stack involves writing low-level CUDA/Triton kernels or wrestling with vLLM & Megatron MoE distributed comms, you'll feel right at home here. We're fully open-source and looking for core contributors.

We'll be dropping the DS V4 roadmap in our GitHub issues over the next week or so, complete with detailed task breakdowns. External contributors are more than welcome to jump in! Grab an issue, join us, and let's ship this together!

👇 Read the Blog :
https://t.co/DqkTYzTZOa

#RLKernel #DeepSeek #AIInfrastructure #RLHF #CUDA #Triton #MoE #RLAlign #ROCm