Udaya Ghai

Applied Scientist @Amazon, PhD in Machine Learning @PrincetonCS

[Zhe Ye @0xlf_]

1/ 🧵Introducing: VeriSpecGen 🚀

Formal verification is a principled way to guarantee code correctness, but writing high-quality specifications remains expensive and expertise-intensive. What if LLMs could reliably synthesize intent-aligned formal specs directly from natural language? Introducing VeriSpecGen, a framework that improves both inference and training time formal specification synthesis performance in Lean! 🛠️✨

📈 SOTA Results: VeriSepcGen achieves 86.6% on the VERINA SpecGen task using Claude Opus 4.5. The framework improves over baselines by up to 31.8pt across different model families and scales, proving robust regardless of base model characteristics.

📄 Paper: https://t.co/IjuiKdbFkX
🔗 Website: https://t.co/ZxTd16n1gQ

[Yuda Song @yus167]

RL on LLMs inefficiently uses one scalar per rollout. But users regularly give much richer feedback: "make it formal," "step 3 is wrong."

Can we train LLMs on this human-AI interaction?

We introduce RL from Text Feedback, with 1) Self-Distillation; 2) Feedback Modeling (1/n) 🧵

[Sadhika Malladi @SadhikaMalladi]

Excited to share that I will be starting as an Assistant Professor in CSE at UCSD (@ucsd_cse) in Fall 2026! I am currently recruiting PhD students who want to bridge theory and practice in deep learning - see here: https://t.co/cHN0Tdr5QQ

[Yuda Song @yus167]

LLMs lose diversity after RL post-training, and this hurts test-time scaling & creativity.
Why does this collapse happen, and how can we fix it?
Our new work introduces:
🔍 RL as Sampling (analysis)
🗺️ Outcome-based Exploration (intervention)
[1/n]

[Rosinality @rosinality]

Outcome-based Exploration for LLM Reasoning

Mitigating reduction of diversity due to RL involves using UCB on answers. There are many studies on this recently (https://t.co/ez9BWWS2lB) and it could be important especially for creative tasks.

[Elad Hazan @HazanPrinceton]

Coming soon to your local neighborhood bookstore :-)
https://t.co/JFc59kPCZv

[Elad Hazan @HazanPrinceton]

*very* excited to share a new *efficient* method for learning *marginally stable* and NONLINEAR dynamical systems, w. brilliant students Evan Dogariu and Anand Brahmbhatt @AnandBrahm15501:
https://t.co/gwVAaPVVst
more info in thread

[Akhil Bagaria @akhil_bagaria]

The deadline for submitting your extended abstracts is fast approaching (Aug 21)! Having a poster will guarantee attendance and you don't want to miss out on what promises to be a really fun day with keynotes from @JohnCLangford @criticalneuro @ben_eysenbach and Ludovic Righetti!

[Udaya Ghai @udayaghai]

@DimitrisPapail Consider if your world is just some relatively easy to compute function and the task is an inverse problem. You understand the world perfectly but computing inverses can be computationally hard.

[Nimit Kalra @ ICML 🇰🇷 @qw3rtman]

Discussing "Mind the Gap" tonight at @haizelabs's NYC AI Reading Group with @leonardtang_ and @willccbb. Authors study self-improvement through the "Generation-Verification Gap" (model's verification ability over its own generations) and find that this capability log scales with pretraining FLOPs 🙌

[Nimit Kalra @ ICML 🇰🇷 @qw3rtman]

Still noodling on this, but the generation-verification gap proposed by @yus167 @_hanlin_zhang_ @ShamKakade6 @udayaghai et al. in https://t.co/zujlBpatj8 is a very nice framework that unifies a lot of thoughts around self-improvement/verification/bootstrapping reasoning

[Kempner Institute at Harvard University @KempnerInst]

4/26 at 10am:

'Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models'

@yus167 · @_hanlin_zhang_ · Carson Eisenach · Sham Kakade · Dean Foster · @udayaghai

Submission: https://t.co/lmqk7jD6rR

[Kempner Institute at Harvard University @KempnerInst]

@_hanlin_zhang_ @johnjvastola @marinkazitnik @nsaphra @ZechenZhang5 4/25 at 10am:

'How Does Critical Batch Size Scale in Pre-training?'

@_hanlin_zhang_ · Depen Morwani · Nikhil Vyas · Jingfeng Wu · Difan Zou · Udaya Ghai · Dean Foster · Sham Kakade

Submission: https://t.co/KxOBkN42sA

[Yuda Song @yus167]

Heading to #ICLR2025 🇸🇬! Excited to connect with friends and chat about RL: theory, LLM reasoning and robotics!

I will present our Oral paper on LLM self-improvement📍4:18pm Sat. Join me if you want to learn about its scaling laws, iterative training and test-time improvement.

[Dylan Foster 🐢 @canondetortugas]

Akshay Krishnamurthy and Audrey Huang (@auddery) have written a nice blog post on the intersection of reinforcement learning theory and language model post-training.

https://t.co/u15MPIMPE7

[Elad Hazan @HazanPrinceton]

Very happy to share the almost-final version of our new online control book:
https://t.co/CklQaFpYcg

[Nikunj Saunshi @NSaunshi]

*New ICLR paper* – We introduce a paradigm of *looped models for reasoning*. Main claims
- Reasoning requires depth (via looping), not necessarily params
- LLM reasoning predictably scales with more loops
- Looped models generate “latent thoughts” & can simulate CoT reasoning
1/n

[Akhil Bagaria @akhil_bagaria]

My 1st last-author paper (joint) will be presented as an Oral at @RealAAAI ! A lot of Option Discovery work (including mine) is based on intuitive heuristics. Instead, we formalize what we want options to do for the agent and then derive algorithms with provable guarantees.