Berk Ustun

Assistant Prof @UCSD. I work on safety, interpretability, and personalization in ML. Previously @GoogleAI @Harvard @MIT @UCBerkeley🇨🇭🇹🇷

[Berk Ustun @berkustun]

Denied a loan by an ML model? You should be able to change something to get approved!

In a new paper w @AlexanderSpangh & @yxxxliu, we call this concept "recourse" & we develop tools to measure it for linear classifiers.

PDF https://t.co/M3dEdEyrXR
CODE https://t.co/9IsXbv6gUt

[Alex Spangher @ Neurips2025 @AlexanderSpangh]

Neurips 2025 was such a blast! We snuck a grand piano into the CreativeAI Track to demo Aria, our pretrained chat-style music model:

[Jessica Hullman @JessicaHullman]

A 'pragmatic intepretability' turn sounds a lot like our argument/framework for evaluating explanation methods--Time to replace task-agnostic fortunetelling w/concrete decision problem specs + theoretic & empirical evidence of expected performance boost
https://t.co/z9XREAdjnt

[Sakana AI @SakanaAILabs]

GPT-5 on Sudoku-Bench 🧩

Since releasing Sudoku-Bench in May 2025, when no LLM could solve a classic 9x9 puzzle, we've been evaluating the latest generation of models. GPT-5 now leads our leaderboard with 33% puzzles solved--approximately 2x the previous leader--and is the first LLM we've tested to solve a 9x9 Sudoku variant.

However, with 67% of the much harder puzzles remaining unsolved, Sudoku-Bench continues to present significant challenges for AI reasoning. Modern Sudoku variants require models to first understand novel rulesets through meta-reasoning, then maintain global consistency across long reasoning chains. Our experiments with GRPO fine-tuning on Qwen2.5-7b and "Thought Cloning" (training on expert human reasoning from Cracking the Cryptic) show that current approaches still struggle with the spatial reasoning and creative "break-in" points that human solvers use naturally. We believe new approaches are required to solve our benchmark.

These results highlight persistent gaps between computational problem-solving and human-like reasoning, particularly in tasks requiring integrated mathematical logic, spatial awareness, and creative insight.

Read more about our update here:
🔗 Blogpost → https://t.co/qLlzYSalUw

[Alex Spangher @ Neurips2025 @AlexanderSpangh]

✨ Very overdue update:

I'll be starting as an Assistant Professor in CS at University of Minnesota, Twin Cities, Fall 2026. I will be recruiting PhD students!!

Please help me spread the word! [Thread] 1/n

[Nick Vincent @nickmvincent]

About a week away from the deadline to submit to the

✨ Workshop on Algorithmic Collective Action (ACA) ✨

https://t.co/fF2sF2S3gw

at NeurIPS 2025!

[Jessica Hullman @JessicaHullman]

I often wonder whether the prospective grad students who contact me understand what they are signing up for. I hope this does the trick

[Hailey Joren @HaileyJoren]

PhD in Computer Science, University of California San Diego 🎓

My research focused on uncertainty and safety in AI systems, including

🤷‍♀️letting models say "I don't know" under uncertainty
🔎understanding and reducing hallucinations
🔁 methods for answering "how much will providing data X improve performance on Y?" at inference time

Many thanks to my advisor @berkustun, to my incredible research collaborators, and to my wonderful friends, husband and family. Getting a PhD while becoming a first-time parent is definitely a recipe for growth!

[Alan Jeffares @Jeffaresalan]

Our new ICML 2025 oral paper proposes a new unified theory of both Double Descent and Grokking, revealing that both of these deep learning phenomena can be understood as being caused by prime numbers in the network parameters 🤯🤯

🧵[1/8]

[Jessica Hullman @JessicaHullman]

Explainable AI has long frustrated me by lacking a clear theory of what explanations should do. Improve use of a model for what? How? Given a task what's max effect explanation can have? It's complicated bc most methods are functions of features & prediction but not true state 1/

[Berk Ustun @berkustun]

Explanations don't help us detect algorithmic discrimination. Even when users are trained. Even when we control their beliefs. Even under ideal conditions... 👇

[Julian Skirzynski @JSkirzynski]

Right to explanation laws assume explanations help people detect algorithmic discrimination.

But is there any evidence for that?

In our latest work w/ David Danks @berkustun, we show explanations fail to help people, even under optimal conditions.

PDF https://t.co/RlKmlzxgxN

[Julian Skirzynski @JSkirzynski]

We’ll be presenting @FAccTConference on 06.24 at 10:45 AM during the Evaluating Explainable AI session!

Come chat with us. We would love to discuss implications for AI policy, better auditing methods, and next steps for algorithmic fairness research.

#AIFairness #xAI

[Julian Skirzynski @JSkirzynski]

Right to explanation laws assume explanations help people detect algorithmic discrimination.

But is there any evidence for that?

In our latest work w/ David Danks @berkustun, we show explanations fail to help people, even under optimal conditions.

PDF https://t.co/RlKmlzxgxN

[Lily Weng @LilyWeng_]

💡LLMs don’t have to be black boxes.

We introduce CB-LLMs -- the first LLMs with built-in interpretability for transparent, controllable, and safer AI.

🚀Our #ICLR2025 paper: https://t.co/08Q4Jcl39L

#TrustworthyAI #ExplainableAI #AI #MachineLearning #NLP #LLM #AIResearch

[Hailey Joren @HaileyJoren]

When RAG systems hallucinate, is the LLM misusing available information or is the retrieved context insufficient? In our #ICLR2025 paper, we introduce "sufficient context" to disentangle these failure modes. Work w J. Zhang, C.S. Ferng, @DaChengJuan1, @ankurtaly @CyrusRashtchian

[Harry Cheon @1000_harrry]

Denied a loan, an interview, or an insurance claim by machine learning models? You may be entitled to a list of reasons.

In our latest w @anniewernerfelt @berkustun @kdphd, we show how existing explanation frameworks can fail and present an alternative tailored for recourse

🧵

[Jessica Hullman @JessicaHullman]

Why is it so hard to show that people can be better decision-makers than statistical models? Some ways that common intuitions about the superiority of human judgment contradict statistical reality, and a few that don't.

https://t.co/5dgGMGuHXc

[Sujay Nagaraj @sujnagaraj]

Many ML models predict labels that don’t reflect what we care about e.g.:
– Diagnoses from unreliable tests
– Outcomes from noisy electronic health records

In our #ICLR2025 paper, we study how this subjects individuals to a lottery of mistakes
Paper: https://t.co/TqQOMTwONy
🧵👇

[Sujay Nagaraj @sujnagaraj]

🚨 Excited to announce a new paper accepted at ICLR2025 in Singapore!

“Learning Under Temporal Label Noise”

We tackle a new challenge in time series ML: label noise that changes over time 🧵👇
https://t.co/Ka7ABXArVR

[Been Kim @_beenkim]

🔥🔥Our small team in Seattle Google DeepMind is hiring! 🔥🔥If you are willing to move to/already in Seattle, has done significant work on human-machine communication / interpretability (from ML side) with a relevant PhD and great publication record, Join us.
Apply here 👉👉 https://t.co/HxU7zIcGMh

[Sanghamitra Dutta @Sangha26Dutta]

Are you interested in serving as a Program Committee member for the ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT 2025)? PC Members are expected to review papers in their area of expertise. Expression of interest form: https://t.co/RufvO5teu0
#FAccT2025