rohin

I build AI for autonomous bots. And music

[rohin @_snackshop_]

@NeelNanda5 As important as safety is, this is math. It’s better that this is out in the public at least. Math is never dangerous.

[rohin @_snackshop_]

Thanks so much and congrats to you too @yuanxue00!!

[Yuan @yuanxue00]

Excited to share that our Enterprise AI team at @scale_AI @ScaleAILabs has two papers accepted to @icmlconf this year.

What makes this especially meaningful is the team (Andrew Klearman Varun Ursekar Apaar Shanker Radu Revutchi Veronica Chatrath Rohin Garg Rishav Chakravarti Sam Denton) behind it. Many of the authors are new and early in their research journey, while at the same time being deeply involved in real customer deployments. This dual lens -- building in production while pushing research forward -- is exactly where I believe the most important innovation is happening. Both works are grounded in a common motivation: agent engineering needs in real-world enterprise settings.

1. VeRO: An Evaluation Harness for Agents to Optimize Agents [https://t.co/uFdTTyjgqa] turns agent self-optimization into a systematic, measurable, and comparable process by introducing a reliable evaluation environment which ensures reproducible results and a benchmark with a standardized suite of target agents, tasks, and evaluation procedures.

2. Coverage, Not Averages: Semantic Stratification for Trustworthy Retrieval Evaluation [https://t.co/l8QvkhgkQY] focuses on trustworthy evaluation. It highlights that the evaluation quality is fundamentally constrained by how evaluation sets are constructed. By introducing semantic stratification, it establishes a formal semantic coverage framework which enables robustness across retrieval regimes and provides interpretable visibility into failure modes.

Proud of the team for pushing on problems that sit at the intersection of research and real-world deployment. This is just the beginning.

#icml #enterpriseAI #reliableAI

[Yuan @yuanxue00]

Excited to share that our Enterprise AI team at @scale_AI @ScaleAILabs has two papers accepted to @icmlconf this year.

What makes this especially meaningful is the team (Andrew Klearman Varun Ursekar Apaar Shanker Radu Revutchi Veronica Chatrath Rohin Garg Rishav Chakravarti Sam Denton) behind it. Many of the authors are new and early in their research journey, while at the same time being deeply involved in real customer deployments. This dual lens -- building in production while pushing research forward -- is exactly where I believe the most important innovation is happening. Both works are grounded in a common motivation: agent engineering needs in real-world enterprise settings.

1. VeRO: An Evaluation Harness for Agents to Optimize Agents [https://t.co/uFdTTyjgqa] turns agent self-optimization into a systematic, measurable, and comparable process by introducing a reliable evaluation environment which ensures reproducible results and a benchmark with a standardized suite of target agents, tasks, and evaluation procedures.

2. Coverage, Not Averages: Semantic Stratification for Trustworthy Retrieval Evaluation [https://t.co/l8QvkhgkQY] focuses on trustworthy evaluation. It highlights that the evaluation quality is fundamentally constrained by how evaluation sets are constructed. By introducing semantic stratification, it establishes a formal semantic coverage framework which enables robustness across retrieval regimes and provides interpretable visibility into failure modes.

Proud of the team for pushing on problems that sit at the intersection of research and real-world deployment. This is just the beginning.

#icml #enterpriseAI #reliableAI

[rohin @_snackshop_]

@gregisenberg Co-author

[Nakshatra @nborgaonkarx]

Elizabeth Taylor’s?? It’s Noor Jahan’s necklace gifted by Jehangir 😭

[rohin @_snackshop_]

Claude code is the closest tech to Jarvis. There’s a long way to go but it’s the closest thing we’ve got and it’s already so beautiful

[rohin @_snackshop_]

reward hacking is crazy
The blue one is without any reference model

[rohin @_snackshop_]

@daniel_mac8 Cos a ton non tech companies have made deals with American fm companies and bought their credits in bulk.

[rohin @_snackshop_]

@naklecha Monarch and TorchForge just landed.

[rohin @_snackshop_]

@Shwinnabego Post drink runs

[rohin @_snackshop_]

@mrsiipa This a trick or a bug

[rohin @_snackshop_]

Pewdiepie moved to grapheneos and is self hosting his note taking app on his steam deck.

[rohin @_snackshop_]

@naklecha Where do you wanna live then. Pls don’t say the Bay Area. That’ll be sad

[rohin @_snackshop_]

@josemorgado Since nadal and Novak aus open

[clem 🤗 @ClementDelangue]

Today, we're unveiling two new open-source AI robots! HopeJR for $3,000 & Reachy Mini for $300. DM me if you want to be added to the waitlist 🤖🤖🤖

Let's go open-source AI robotics!

[Younggyo Seo @younggyoseo]

Excited to present FastTD3: a simple, fast, and capable off-policy RL algorithm for humanoid control -- with an open-source code to run your own humanoid RL experiments in no time!

Thread below 🧵

[rohin @_snackshop_]

@GoodfireAI this is awesome, thanks for opensourcing it all! Gonna play around with the SAEs now

[rohin @_snackshop_]

@mrsiipa I’d say you’re undervaluing everything you’ve learnt before this month. Huge recency bias.

[rohin @_snackshop_]

@POSTWOOK @Microsoft not buying this would be stupid

[rohin @_snackshop_]

@Pirat_Nation Rage button it is rhen