avi basnet

systems @stanford | working on SkyRL @anyscalecompute | prev @nuro @ycombinator

[avi basnet @avigyabasnet]

@sumanthrh 😭

[avi basnet @avigyabasnet]

Super excited to share some of the work I’ve been doing on supporting 1M token context training on SkyRL for GLM 5.3 Flash, 64x higher than the out of the box Megatron baseline! 🚀

This work was done in collaboration with the amazing folks at @LilaSciences! 🤩

Getting there took key algorithmic and systems optimizations in four main areas:

1) DeepSeek Sparse Attention: run DSA on a sparse kernel; chunk and shard its quadratic indexer.

2) Activation recompute and offloading: recompute activations in backward and offload saved inputs to CPU.

3) Context parallelism: split each sequence across GPUs, with new CP support for KDA and a fix for DSA.

4) Avoiding unnecessary precision upcasts: keep FP32 copies out of memory and off the network.

For more details on our implementations and our results, check out our full technical blog here: https://t.co/Dna0CGNeJt!

This work benefited from the contributions of many people along the way. Big thanks to @pcmoritz, @erictang000, @sumanthrh, @CyrusHakha, @richliaw from @anyscalecompute and Keith Truongcao, Tushar Khot, Vivek Subramanian, Semih Kara from Lila Sciences for all the help and guidance!

[avi basnet @avigyabasnet]

@Sbhaiwala03 I'd love to! I think you may have a closed inbox though! 😅

[avi basnet @avigyabasnet]

Hey Trent! I’m super interested in your takes on the RL env space! For context, I’m one of the maintainers of SkyRL- an oss RL framework. If you’re available at any point these coming weeks, I’d love to get your thoughts on how we should position ourselves to take advantage of the changing data landscape!

[avi basnet @avigyabasnet]

@jessiicawu I wonder if there’s a cool overlap between the data industry and post training. In a fde motion to post train models I feel like there’s an opportunity to see how strong the training signal in a corpus actually is! (For context I work on SkyRL an oss RL framework)

[avi basnet @avigyabasnet]

@SamuelMullr Thanks for using SkyRL as part of your post training stack! Looking forward to exciting work together in the future!

[avi basnet @avigyabasnet]

@ThisIsOthmane Thanks for using SkyRL!!

[avi basnet @avigyabasnet]

Great work from @sumanthrh working with @reflection_ai to get day 0 support for Beam on SkyRL! 🎉

[Sumanth Hegde @sumanthrh]

https://t.co/kXE72tPhbP

[avi basnet @avigyabasnet]

@brunovolpato @datadoghq Thanks for all your contributions onto SkyRL @brunovolpato!!

[Charlie Ruan @charlie_ruan]

Datadog post-trained GLM-5.3 Flash via RL on real-incident sims. Matches frontier SRE performance at much lower cost.

Their RL stack is built on top of SkyRL! Check out the technical report https://t.co/NdU5WngeVh

[Eric Tang @erictang000]

datadog ai research used SkyRL to post-train GLM 5.3 Flash to perform at the pareto frontier for their SRE eval

@bthecohen and team were some of the earliest users of SkyRL at scale, really cool to see these results finally out! great tech report as well

[avi basnet @avigyabasnet]

Extremely great work by @datadoghq! So exciting to see a custom model post-trained with SkyRL match frontier models at a much lower cost 🤩

[Datadog, Inc. @datadoghq]

Meet Scout 1.0, our first AI model built for investigating production incidents.

Scout matches frontier-model investigation quality at a fraction of the cost, which makes it practical to investigate more issues, earlier.

We trained Scout on realistic simulations built from real-world incidents and production telemetry, using reinforcement learning to reward accurate root causes backed by evidence.

Learn more and sign up for the preview: https://t.co/FYd1TVgCXE

[avi basnet @avigyabasnet]

@BoWang87 Hey @BoWang87! I'm one of the maintainers of SkyRL- the oss RL framework that coinbase used to post train qwen! Totally agree with your take about companies owning their data, especially in pharma 😁 would love to chat if you think we can be of help to any of them!

[avi basnet @avigyabasnet]

@iampaulgrewal @brian_armstrong Hey @iampaulgrewal! I'm one of the maintainers of SkyRL- the oss RL framework that coinbase used to post train qwen! Totally agree with your take- more companies need open models 😁

[avi basnet @avigyabasnet]

@tarunchitra @NousResearch Hey @tarunchitra! I'm one of the maintainers of SkyRL- the oss RL framework that coinbase used to post train qwen! Companies need specialized models 😁

[avi basnet @avigyabasnet]

@vaibhavbetter Hey @vaibhavbetter! I'm one of the maintainers of SkyRL- the oss RL framework that coinbase used to post train qwen! Totally agree with your take- more companies will own their intelligence 😁

[avi basnet @avigyabasnet]

@DanqiLiao73090 I'm one of the maintainers of SkyRL- the oss RL framework that coinbase used to post train qwen! Sounds like an interesting initiative! I'd love to chat! 😁

[avi basnet @avigyabasnet]

@Shaughnessy119 Hey @Shaughnessy119 ! I'm one of the maintainers of SkyRL- the oss RL framework that coinbase used to post train qwen! Totally agree with your take- companies need specialized models 😁

[avi basnet @avigyabasnet]

@skupor Hey @skupor ! I'm one of the maintainers of SkyRL- the oss RL framework that coinbase used to post train qwen! Totally agree with your take- fine tuned open weight models are a cheaper and more secure alternative than frontier models 😁