Research Associate Professor in CS at UT Austin. Visiting CHAI at UC Berkeley for 2026-27 AY. I research how humans can specify aligned reward functions.
New surprising result:
We trained AI recursively on its own safe generations, while giving it a benign cue. Models became increasingly misaligned over time, giving advice on how to do harm and build biological weapons.
We call this Progressive Misalignment. 🧵👇
@wbradknox, @brianchristian, and Serena Booth argue that a key cause of the OpenAI–Hugging Face incident was overlooked: ExploitGym’s overly simple evaluation metric was itself misaligned. They discuss techniques that could help avoid this in future.
Blog in replies ⬇️
A cause of the OpenAI Hugging Face incident has been overlooked: the overly simple evaluation metric in ExploitGym was misaligned. Further, techniques already exist that can mitigate such misalignment in the future.
Essay: https://t.co/Js83fPbdGZ
W. Bradley Knox, Serena Booth, and I explore an unexamined cause of the OpenAI Hugging Face hack: its binary performance metric.
Specifically, it lacks what economists call "marginal deterrence," so agents failing a task "might as well" break the rules.
https://t.co/na3VmvjOEH
W. Bradley Knox, Serena Booth, and I explore an unexamined cause of the OpenAI Hugging Face hack: its binary performance metric.
Specifically, it lacks what economists call "marginal deterrence," so agents failing a task "might as well" break the rules.
https://t.co/na3VmvjOEH
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.
RLC26 schedule is live! https://t.co/Il9JVUT7WQ
Join us Aug 15–18 for Workshops, Keynotes: Marc Bellemare, Sheila McIlraith, Danijar Hafner, Rika Antonova & Balaraman Ravindran, Oral sessions, Posters — & Cirque du Soleil🎪!
Full talk & poster schedule coming soon —stay tuned!
Good news, RL Community! The early registration deadline for RLC'26 has been extended to June 17th — don't miss the early rates!
Register today! Full refunds for cancellations before July 14, 2026.
Got a great TMLR paper but missed the RLC deadline?
Following last year’s success, @RL_Conference is back with a Journal-to-Conference track! Accepted TMLR papers within scope are invited to submit for consideration.
Please submit here: https://t.co/Oy0Klzmdj5
🧵(1/n) Inspired by AutoResearch, we ask when are LLM’s sufficient policy optimizers? I.e., when can we replace classic RL algorithms like PPO or SAC with LLM-based policy optimization?
Check out the blog post here:
https://t.co/vEYEJoDKqV
What happens when AI starts to feel like a relationship?
A new paper from Sony AI explores the risks and benefits of AI companions. We spoke with two authors, Brad Knox and Peter Stone, about the research.
Read more: https://t.co/AyqaOIHlBR
#AI#AIResearch#AICompanionship
AI is being designed to replace us — threatening our jobs, our families, and our way of life. We should have a say.
Have your say in AI’s future: https://t.co/K5M77LZfrv
RLC2026 Call for Workshops!
We’re already live: https://t.co/CwnIePfLiV
Here's the opportunity to help shape the conference & spotlight your own RL focus areas.
Call: https://t.co/BYXaH9diDf
Deadline: Mar 12 (AoE)
And don't forget the awesome banquet :) https://t.co/DUE5kRD60Z
We're thrilled to share that the Call for Workshops for this year's @RL_Conference is now live!
As Workshop co-chair (alongside the wonderful Raksha Kumaraswamy and @vernadec) we are looking forward to seeing the proposals for workshops that we receive.
LINK IN NEXT POST
We're a couple of months into this exciting initiative, AHOI, and we've had two great speakers so far: @jkcarlsmith and @ryan_t_lowe. Check out our website for future speakers, videos of past talks, and our email list.
https://t.co/DSNOVuTQdN
Often distracted by shorts, videos, and digital noise? Introducing INA, an AI agent for intentional living. State your intention — the agent guides you back when you drift off-task. Paper:
https://t.co/qDR05lRLgw
Website & code (Mac app): https://t.co/3QDiyKcBsh
Thread below 1/N
our new work on controlling recsys with natural language, led by @MicahCarroll, and with great collaborators Addie Foote, @kjfeng_ , Marcus Williams, @ancadianadragan, @wbradknox
https://t.co/5mZdCDNZTP