NovaSky

Building SkyRL at @BerkeleySky
Join the Slack community:

[NovaSky @NovaSkyAI]

We are excited to announce that SkyRL now implements the Tinker API. Run Tinker training scripts on your own hardware with zero code changes.

Try it out today: https://t.co/uD5DlDjhnp

[Tyler Griggs @tyler_griggs_]

SkyRL now implements the Tinker API.

Now, training scripts written for Tinker can run on your own GPUs with zero code changes using SkyRL's FSDP2, Megatron, and vLLM backends.

Blog: https://t.co/GAtW81jM38
🧵

[Sumanth Hegde @sumanthrh]

Great to see this out!

Datadog's new SOTA model for investigating production incidents. Their in-house training stack builds on top of SkyRL! 🚀🚀🚀

[Eric Tang @erictang000]

datadog ai research used SkyRL to post-train GLM 5.3 Flash to perform at the pareto frontier for their SRE eval

@bthecohen and team were some of the earliest users of SkyRL at scale, really cool to see these results finally out! great tech report as well

[Charlie Ruan @charlie_ruan]

Datadog post-trained GLM-5.3 Flash via RL on real-incident sims. Matches frontier SRE performance at much lower cost.

Their RL stack is built on top of SkyRL! Check out the technical report https://t.co/NdU5WngeVh

[Charlie Ruan @charlie_ruan]

Post-trained with SkyRL!

[Richard Liaw @richliaw]

Post-trained with SkyRL @NovaSkyAI

Served on Ray Serve LLM @raydistributed

all on Anyscale Platform @anyscalecompute

[AfterQuery @AfterQuery]

Read more about our experiment using SkyRL here: https://t.co/LOVN81MrDE

To learn more about our tasks and reward design, reach out to us at [email protected]

[Casper Hansen @casper_hansen_]

Kimi K2.5-2.7 can be trained with RL on just 16 B300 GPUs (yes, just 2 nodes!) with full context

Most post-training frameworks out there will tell you that you need 16 full nodes, and might not even give you full context

Thanks to SkyRL maintainers for collab

[Eric Tang @erictang000]

Excited to announce the release of SkyRL v0.4.0!

Some highlights:
- Support for GLM 5.3 Flash, GLM 5.3, and Kimi K2.7
- Top-P sampler replay and optimized R3 data transfer
- Full FP8/MXFP8 RL
- Sharded P2P weight transfer

Blog: https://t.co/74zIkEWIDj

Details in 🧵

[NovaSky @NovaSkyAI]

SkyRL v0.4.0 enables large scale RL training with features like top-p sampler replay, R3, and FP8 RL on new models like GLM 5.3 Flash from @Zai_org!

[Eric Tang @erictang000]

Excited to announce the release of SkyRL v0.4.0!

Some highlights:
- Support for GLM 5.3 Flash, GLM 5.3, and Kimi K2.7
- Top-P sampler replay and optimized R3 data transfer
- Full FP8/MXFP8 RL
- Sharded P2P weight transfer

Blog: https://t.co/74zIkEWIDj

Details in 🧵

[Edward Hu @edwardjhu]

How does one RL post-train a 397B model for long-horizon knowledge work? 👩‍💼

We share every step we took to bring Qwen 3.5 397B from 16.1% Pass@1 to 27.3% on APEX-Agents using DPPO, including final models weights and the full training script🚀 This is the first of many works from Mercor Research on open model training research.

Full blog: https://t.co/GWgR2DB7iy
Source code: https://t.co/96h8ADfvBp

[Charlie Ruan @charlie_ruan]

Worked with Mercor Research and the SkyRL team, training Qwen3.5-397B-A17B on APEX-Agents (long-horizon office work) off-the-shelf data with SkyRL, improving Pass@1 from 16% to 27%.

The post is more of a practical field guide for what to do given an RL dataset, de-risking step by step: environment infra, harness fixes/optimizations, token-in-token-out, an overfitting run, small-model ablations, then the hero run.

We were also delighted to see the gains transfer to OpenCode despite training on an MCP-based harness, and even to Terminal-Bench 2.1.

Code, models, and eval traces are all open source. Hope it helps your next hillclimb!

[vLLM @vllm_project]

In RL training, a vLLM rollout engine and a Megatron trainer can run the same policy yet disagree on a token's logprob due to floating-point non-associativity.

SkyRL's IsoExec combines an execution contract with a unified model, aligning rounding-sensitive execution choices across rollout and training.

Bitwise parity holds across different TP, EP, and SP layouts. For Gated DeltaNet, the chunkwise-parallel recurrent algorithm makes parallel training and prefill bitwise identical to recurrent decode.

Qwen3.5-35B-A3B, DAPO, 8xH100, 50 steps: logprob diff 1.6e-2 to 6.7e-7, full-step overhead 25.3% ✅ vLLM's scheduler and CUDA graphs still apply.

Thanks to @JiangAlexander1 and the SkyRL team at @NovaSkyAI.

🔗 https://t.co/Y8SIIgHqAr

[NovaSky @NovaSkyAI]

It was a packed house at the SkyRL Meetup today.

Thanks to all the speakers!

[Sumanth Hegde @sumanthrh]

Join us for the first-ever SkyRL Meetup!

It's been a year since we released the SkyRL library. SkyRL now has 2k+ GitHub stars and a growing user base in academia and industry, including researchers from Stanford, CMU, UC Berkeley, Microsoft AI, Datadog, and more.

We're hosting a SkyRL Meetup on August 18th at the Anyscale office in SF. We have a special set of lightning talks:

🔗 Making frontier RL training accessible to everyone, from @trajectorylabs
🐶 Post-training an SRE agent with SkyRL, from @datadoghq
⚡️ Scaling RL with SkyRL on AMD GPUs, from @AMD
🧑🏻‍💻 Training knowledge work agents with SkyRL, from the SkyRL team (@charlie_ruan)

We’ll also do a deep dive into the library and talk about the road ahead. More details here: https://t.co/UOWfaaWRIU

[Charlie Ruan @charlie_ruan]

The first-ever SkyRL Meetup is happening! A year of hard work from the team, and it's been great to see users pick it up from across industry and academia.

Join us Aug 18 Tuesday 6:00pm in SF for four lightning talks from enterprise users and collaborators (@trajectorylabs , @datadoghq , @AMD , @mercor ), followed by a SkyRL technical deep dive.

Would love to see you there!

[Sumanth Hegde @sumanthrh]

Join us for the first-ever SkyRL Meetup!

It's been a year since we released the SkyRL library. SkyRL now has 2k+ GitHub stars and a growing user base in academia and industry, including researchers from Stanford, CMU, UC Berkeley, Microsoft AI, Datadog, and more.

We're hosting a SkyRL Meetup on August 18th at the Anyscale office in SF. We have a special set of lightning talks:

🔗 Making frontier RL training accessible to everyone, from @trajectorylabs
🐶 Post-training an SRE agent with SkyRL, from @datadoghq
⚡️ Scaling RL with SkyRL on AMD GPUs, from @AMD
🧑🏻‍💻 Training knowledge work agents with SkyRL, from the SkyRL team (@charlie_ruan)

We’ll also do a deep dive into the library and talk about the road ahead. More details here: https://t.co/UOWfaaWRIU

[Sumanth Hegde @sumanthrh]

Excited to announce the release of SkyRL v0.3.0!

SkyRL’s focus in this release revolves around three major themes:
1. Support for large scale, async RL
2. Improving SkyRL’s Tinker API for self-hosting Tinker servers
3. Providing a unified library for SFT and RL

Some highlights 🧵

[Kexin Huang @KexinHuang5]

Today, we're excited to share that Biomni is published in @ScienceMagazine.

Biomedical research is still fragmented, manual, and difficult to scale. In this work, we introduce Biomni - the first general-purpose biomedical AI agent with an integrated biology environment that can reason, plan, and execute end-to-end scientific workflows.

We show that, with the right environment and harness, AI can automate large-scale omics analyses, orchestrate laboratory robotics, optimize molecular properties, and even train new AI models for biology.

We also introduce a reinforcement learning recipe for continually improving biomedical AI agents, enabling open-source models to achieve frontier-level performance.

It's surreal to look back. We started the Biomni project in early 2024, when agentic AI was still nascent. It is exciting to see tens of thousands of biologists collaborating with agents every day to accelerate science.

Try Biomni: https://t.co/Pzm8TVlkNA
Read more: https://t.co/pRstxzPKDi

This work is not possible without this truly inter-disciplinary team: @serena2z @hcwww_ @YuanhaoQ Minta Lu, Ryan Li, @yusufroohani Lin Qiu @shiyi_c98 Gavin Junze Di @rickwierenga @kavi_deniz Sherry @TianweiShe Shruti Jennefer Xin Zhou @MWheelerMD Jon Bernstein @MengdiWang10 @PengHeAtlas @zhou_jingtian @SnyderShot @lecong Aviv Regev @jure

@StanfordAILab @genentech @phylo_bio @arcinstitute @UW @berkeley_ai @RetroBio_ @tamarindbio @Princeton @UCSF

[Shiyi Cao @shiyi_c98]

Biomni is now out in Science @ScienceMagazine ! Huge congrats to the team 🎉

Biomni is a really exciting step toward AI agents that can carry out real biomedical research workflows. Proud that SkyRL supported the RL training for Biomni-R0 early last year, where we explored end-to-end multi-turn RL for biomedical reasoning agents.

It’s really exciting to see how AI agents have evolved the past one year toward carrying out real scientific workflows and accelerating discovery.

[Casper Hansen @casper_hansen_]

SkyRL Tinker implementation is so cool, but still early days so I went ahead and implemented a basic chunked cross-entropy for 23x less memory usage!

Now most models fit much more seamlessly in memory when combined with Megatron.

[Stas Bekman @StasBekman]

In parallel we announce a new open source repo:

https://t.co/8wNDXCzIhQ

This is the framework for very fast RL (and future other optimizations rolled into it)

It currently has all the code you need to use or integrate Arctic RL into RL frameworks, with SkyRL and Verl available and more framework integrations coming.

Please kindly spread the word! Thank you!