uccl_project

Building the next-generation AI/ML networking

[Yang Zhou @yangzhouy]

🎯 Hey, here is an in-depth analysis of the recent OpenAI's MRC (multipath reliable connection) protocol for GPU RDMA networking.

https://t.co/E4IjdpSHQ5

[Ziming Mao @ziming_mao]

🚀 Excited to release mKernel: a set of fast multi-node, multi-GPU fused kernels.

💻 Code: https://t.co/y2WfdMVTfC
📝 Blog: https://t.co/wGomxmeRxr

mKernel fuses compute + communication into one persistent GPU kernel, covering both intra/inter-node with GPU-initiated communication.

Amazing team: @yangzhouy, Chon Lam Lao, Costin Raiciu, Scott Shenker, @istoica05

[uccl_project @uccl_proj]

Check out this practitioner guide to AWS EFA RDMA programming from the UCCL team! https://t.co/meQGeJSc2H

EFA ordering and atomics are different from other NIC vendors, so the UCCL project (https://t.co/4kyjuItIBy) takes a special approach to support applications such as DeepEP.

[uccl_project @uccl_proj]

If you want to run @vllm_project or @sgl_project expert parallelism on heterogeneous hardware, try UCCL-EP!

It has supported both inference frameworks, and also Megatron-LM for training!

[Ziming Mao @ziming_mao]

🚀 Excited to share the training & inference results for UCCL-EP: a portable, high-performance expert-parallel communication library across heterogeneous GPU + NIC hardware.
💻 Code: https://t.co/wVWiso8ajS
📝 Blog: https://t.co/PSRH7CQpK6
📈 Highlights:
• Up to 45% faster Megatron-LM training vs RCCL on 128 AMD GPUs
• Up to 40% faster SGLang inference vs NCCL on 32 H200 GPUs
• Up to 25% lower vLLM TPOT vs NCCL
• Up to 2.3x better EP dispatch/combine on AWS EFA
🔁 Fully portable across heterogeneous GPU/NIC hardware and a drop-in replacement for DeepEP

Amazing team: Chon Lam Lao, @yangzhouy, Yihan Zhang, Chihan Cui, Zhongjie Chen, Zhiying Xu, @KaichaoYou, Zhen Huang, Zhenyu Gu, Costin Raiciu, Scott Shenker, @istoica05

[Ziming Mao @ziming_mao]

🚀 Excited to share the training & inference results for UCCL-EP: a portable, high-performance expert-parallel communication library across heterogeneous GPU + NIC hardware.
💻 Code: https://t.co/wVWiso8ajS
📝 Blog: https://t.co/PSRH7CQpK6
📈 Highlights:
• Up to 45% faster Megatron-LM training vs RCCL on 128 AMD GPUs
• Up to 40% faster SGLang inference vs NCCL on 32 H200 GPUs
• Up to 25% lower vLLM TPOT vs NCCL
• Up to 2.3x better EP dispatch/combine on AWS EFA
🔁 Fully portable across heterogeneous GPU/NIC hardware and a drop-in replacement for DeepEP

Amazing team: Chon Lam Lao, @yangzhouy, Yihan Zhang, Chihan Cui, Zhongjie Chen, Zhiying Xu, @KaichaoYou, Zhen Huang, Zhenyu Gu, Costin Raiciu, Scott Shenker, @istoica05

[uccl_project @uccl_proj]

11/ 🤝 Collaborators: @ziming_mao, @yangzhouy, Yihan Zhang, Chihan Cui, Zhongjie Chen, Zhiying Xu,
@KaichaoYou, Zhen Zhang, Zhenyu Gu, Costin Raiciu, Scott Shenker, @istoica05 from @Berkeley_EECS, @ucdavis, @UWMadison, @Tsinghua_Uni, @awscloud, @AMD, @Broadcom, UPB

[uccl_project @uccl_proj]

🚀 Introducing UCCL-EP: A portable, efficient Expert Parallelism framework that brings DeepEP-level GPU-driven communication with the same APIs to any cloud or hardware — AWS EFA, AMD GPUs, Broadcom NICs and beyond.
Blog: https://t.co/d3oBVlWezZ
Code: https://t.co/0UbCUYz9N9

[uccl_project @uccl_proj]

10/ 🌍 Vision: UCCL-EP democratizes expert-parallel training — GPU-driven performance without vendor lock-in.

[uccl_project @uccl_proj]

9/N🤝Moreover, we provide free consulting on performance-related NCCL/RCCL issues, so reach out to us if you hit any such issues.

[uccl_project @uccl_proj]

1/N📢 Debugging NCCL performance problems for LLM workloads is always challenging. In this blog post, we explore various perf-critical parameters in NCCL and tackle datacenter network congestions with UCCL plugin.
https://t.co/zN49roaP6t

[uccl_project @uccl_proj]

8/N🌟UCCL is fully open-source at https://t.co/4kyjuItIBy, with many developers and maintainers from UC Berkeley Sky Computing Lab, the lab that created Spark, Ray, and vLLM. We enthusiastically invite open-source developers to join and contribute to the UCCL project.