NovaSky

Building SkyRL at @BerkeleySky
Join the Slack community:

[NovaSky @NovaSkyAI]

We are excited to announce that SkyRL now implements the Tinker API. Run Tinker training scripts on your own hardware with zero code changes.

Try it out today: https://t.co/uD5DlDjhnp

[Tyler Griggs @tyler_griggs_]

SkyRL now implements the Tinker API.

Now, training scripts written for Tinker can run on your own GPUs with zero code changes using SkyRL's FSDP2, Megatron, and vLLM backends.

Blog: https://t.co/GAtW81jM38
🧵

[Bryan Lu @yuzhe_lu]

Great to see @modal launch this! You can also deploy a Tinker endpoint directly on AWS EC2 with SkyRL @NovaSkyAI for the same cost and flexibility benefits, and start agentic RL training with our AgentCore RL Toolkit! https://t.co/NPiscXoDBM

[Qiuyang Mang @MangQiuyang]

FrontierCS 2 is our new benchmark for CS research & development.

Each task is based on a paper: we delete the contribution from the released code, and ask agents to outdo the original method, as judged by the authors' chosen metrics. Agents don't have access to the paper or web.

[SkyPilot @skypilot_org]

Post-training a 4B model with RL to fly Tesla Roadster. It outperforms recent open models and even GPT-6 Astra in a storm.

The key: have the model learn from the teachers first and then learn itself in practice.

Trained with SkyRL (@NovaSkyAI) and vLLM (@vllm_project), orchestrated by SkyPilot's Job Group (@skypilot_org).

Check it out ⬇️

[avi basnet @avigyabasnet]

Super excited to share some of the work I’ve been doing on supporting 1M token context training on SkyRL for GLM 5.3 Flash, 64x higher than the out of the box Megatron baseline! 🚀

This work was done in collaboration with the amazing folks at @LilaSciences! 🤩

Getting there took key algorithmic and systems optimizations in four main areas:

1) DeepSeek Sparse Attention: run DSA on a sparse kernel; chunk and shard its quadratic indexer.

2) Activation recompute and offloading: recompute activations in backward and offload saved inputs to CPU.

3) Context parallelism: split each sequence across GPUs, with new CP support for KDA and a fix for DSA.

4) Avoiding unnecessary precision upcasts: keep FP32 copies out of memory and off the network.

For more details on our implementations and our results, check out our full technical blog here: https://t.co/Dna0CGNeJt!

This work benefited from the contributions of many people along the way. Big thanks to @pcmoritz, @erictang000, @sumanthrh, @CyrusHakha, @richliaw from @anyscalecompute and Keith Truongcao, Tushar Khot, Vivek Subramanian, Semih Kara from Lila Sciences for all the help and guidance!

[Zhanghao Wu @Michaelvll1]

Check our blog: https://t.co/HQ4dN5SoRd

Trained with SkyRL (@NovaSkyAI), served with vLLM (@vllm_project), and orchestrated with SkyPilot (@skypilot_org)

[Zhanghao Wu @Michaelvll1]

Tesla's flying Roadster went viral. The real demo had been delayed two weeks "due to high winds."

We asked: could the latest LLMs sit in the pilot's seat and fly the car through that wind?

We built an RL environment for LLMs to fly it. In real time, GPT-6 Astra, DeepSeek, Muse Spark Open, Beam and Nemotron 3.5 Super VL landed 0% of kickflips.

Even with unlimited time to think for each decision, they still did worse than a Qwen 4B model we post-trained with RL in a storm.

[Reflection @reflection_ai]

Model customization recipes are live on our website and in the cookbook repo.

Beam is now supported across RL and SFT stacks including @axolotl_ai, NeMo RL, NeMo AutoModel, Megatron Bridge (@NVIDIAAI), Miles (@radixark), SkyRL (@NovaSkyAI), and TRL (@huggingface).

Get cooking 🧑‍🍳
https://t.co/gRHcqEsa0s

[Charlie Ruan @charlie_ruan]

First day 0 model support on SkyRL. Great work from @sumanthrh !

[vLLM @vllm_project]

🎉 Day-0 RL support for Beam in SkyRL @NovaSkyAI.

LoRA, BF16 full fine-tuning and MXFP8 RL, with vLLM rollouts and Reflection’s Megatron-Bridge implementation.

SkyRL reports +62% LoRA decode throughput from right-sizing expert adapters in vLLM: 276 → 447 tok/s/GPU at 32 concurrent requests, PP2×TP8. This is a decode result, not an end-to-end training speedup.

R3 works with pipeline parallelism, and FP32 router weights are preserved during weight sync. Thanks to Sumanth Hegde and the SkyRL team!

https://t.co/suzHf6x7N5

[Sumanth Hegde @sumanthrh]

https://t.co/kXE72tPhbP

[Sumanth Hegde @sumanthrh]

Great to see this out!

Datadog's new SOTA model for investigating production incidents. Their in-house training stack builds on top of SkyRL! 🚀🚀🚀

[Eric Tang @erictang000]

datadog ai research used SkyRL to post-train GLM 5.3 Flash to perform at the pareto frontier for their SRE eval

@bthecohen and team were some of the earliest users of SkyRL at scale, really cool to see these results finally out! great tech report as well

[Charlie Ruan @charlie_ruan]

Datadog post-trained GLM-5.3 Flash via RL on real-incident sims. Matches frontier SRE performance at much lower cost.

Their RL stack is built on top of SkyRL! Check out the technical report https://t.co/NdU5WngeVh

[Charlie Ruan @charlie_ruan]

Post-trained with SkyRL!

[Richard Liaw @richliaw]

Post-trained with SkyRL @NovaSkyAI

Served on Ray Serve LLM @raydistributed

all on Anyscale Platform @anyscalecompute

[AfterQuery @AfterQuery]

Read more about our experiment using SkyRL here: https://t.co/LOVN81MrDE

To learn more about our tasks and reward design, reach out to us at [email protected]

[Casper Hansen @casper_hansen_]

Kimi K2.5-2.7 can be trained with RL on just 16 B300 GPUs (yes, just 2 nodes!) with full context

Most post-training frameworks out there will tell you that you need 16 full nodes, and might not even give you full context

Thanks to SkyRL maintainers for collab

[Eric Tang @erictang000]

Excited to announce the release of SkyRL v0.4.0!

Some highlights:
- Support for GLM 5.3 Flash, GLM 5.3, and Kimi K2.7
- Top-P sampler replay and optimized R3 data transfer
- Full FP8/MXFP8 RL
- Sharded P2P weight transfer

Blog: https://t.co/74zIkEWIDj

Details in 🧵

[NovaSky @NovaSkyAI]

SkyRL v0.4.0 enables large scale RL training with features like top-p sampler replay, R3, and FP8 RL on new models like GLM 5.3 Flash from @Zai_org!

[Eric Tang @erictang000]

Excited to announce the release of SkyRL v0.4.0!

Some highlights:
- Support for GLM 5.3 Flash, GLM 5.3, and Kimi K2.7
- Top-P sampler replay and optimized R3 data transfer
- Full FP8/MXFP8 RL
- Sharded P2P weight transfer

Blog: https://t.co/74zIkEWIDj

Details in 🧵

[Edward Hu @edwardjhu]

How does one RL post-train a 397B model for long-horizon knowledge work? 👩‍💼

We share every step we took to bring Qwen 3.5 397B from 16.1% Pass@1 to 27.3% on APEX-Agents using DPPO, including final models weights and the full training script🚀 This is the first of many works from Mercor Research on open model training research.

Full blog: https://t.co/GWgR2DB7iy
Source code: https://t.co/96h8ADfvBp