I'm excited to share that https://t.co/eDFaNGJ0TF has been accepted to NeurIPS 2026! 🎉
Huge thanks to my co-lead @Yiming_Su3, our co-authors @SaadMRP, @lilygn6, Yifan Tian, Hans-Arno Jacobsen, @yinfang_chen , @TianyinXu, and our 60+ contributors!
We have benchmarks for agents that write code.
We built one for what happens after you deploy it.
Can AI resolve production issues? 🧵 [1/N]
Super excited to be presenting https://t.co/HdYZGstPVj at @SnorkelAI's Frontier Data Summit, Oct 8 in SF. Can AI really solve production issues? Come find the poster. Request an invite: https://t.co/drAHbW8nOU
Frontier Data Summit: Oct 8, SF. @fchollet, @sanmikoyejo, @nikogrupen, @ajratner, @alexgshaw, @StevenDillmann, and others onstage plus 25+ posters with the teams behind Agent's Last Exam, OSWorld 2.0, Terminal Bench Science, Terminal Bench, CollusionBench, PostTrainBench, T² Scaling Laws, SlopCodeBench, LONGRUN, SREGym, BizBench, and more.
Seeing the PR to #SREGym by @niallm, an author of the holy Google SRE book (@srebook) and the Reliable Machine Learning book is rather rewarding.
The PR is very SRE style 🙂
https://t.co/oLyDk6xcPv
Niall, now a Distinguished Engineer @ciroosai, is also a great, patient mentor for students on the #SREGym Slack.
The SREGym team (me, @HacksonClark , @lilygn6 in person, @SaadMRP remotely) will be giving a demo at CAIS @CAISconf TODAY at 4:30 PM in San Jose room! Come to chat with us about AI SRE, benchmarking, and more!
Bonus: if you have a laptop, we have a self-contained demo/artifact for you to try without LLM credits here: https://t.co/eAhMIy3AJu.
Arrived at @CAISconf today and started the conference. I got to catch up with a few friends (Hi @LakshyAAAgrawal@melissapan@mertcemri@JonSaadFalcon and other folks!) at the Laude Lounge. It's always been great to sync up our research progress and exchange ideas about future directions. Huge thanks to @LaudeInstitute for organizing the Laude Lounge again!!!
Surprisingly, there are a few people that also work in AI+SRE space which I got to talk to and share what we do at SREGym. It's so encouraging that people were like "Yes please! Being able to simulate production failures is so cool! What failures do you simulate? How does frontier models perform on it?"
I got to meet my hero today: I talked to Dave Patterson about a historical project he worked on, called Recovery-Oriented Computing, which is a key inspiration for our work on Stratus (https://t.co/fPEOHveTi2), and a very personal inspiration to be a systems reliability researcher. The papers in the project make me believe that reliability is a design problem, not a {language, framework, architecture, etc.} problem. He shared a few insights into reliability research in the agentic era, which I now have to spend days and nights thinking about.
I will be in the bay area until Sunday! Drop me a DM if you want to chat/grab coffee together!