Denied a loan by an ML model? You should be able to change something to get approved!
In a new paper w @AlexanderSpangh & @yxxxliu, we call this concept "recourse" & we develop tools to measure it for linear classifiers.
PDF https://t.co/M3dEdEyrXR
CODE https://t.co/9IsXbv6gUt
A 'pragmatic intepretability' turn sounds a lot like our argument/framework for evaluating explanation methods--Time to replace task-agnostic fortunetelling w/concrete decision problem specs + theoretic & empirical evidence of expected performance boost
https://t.co/z9XREAdjnt
GPT-5 on Sudoku-Bench 🧩
Since releasing Sudoku-Bench in May 2025, when no LLM could solve a classic 9x9 puzzle, we've been evaluating the latest generation of models. GPT-5 now leads our leaderboard with 33% puzzles solved--approximately 2x the previous leader--and is the first LLM we've tested to solve a 9x9 Sudoku variant.
However, with 67% of the much harder puzzles remaining unsolved, Sudoku-Bench continues to present significant challenges for AI reasoning. Modern Sudoku variants require models to first understand novel rulesets through meta-reasoning, then maintain global consistency across long reasoning chains. Our experiments with GRPO fine-tuning on Qwen2.5-7b and "Thought Cloning" (training on expert human reasoning from Cracking the Cryptic) show that current approaches still struggle with the spatial reasoning and creative "break-in" points that human solvers use naturally. We believe new approaches are required to solve our benchmark.
These results highlight persistent gaps between computational problem-solving and human-like reasoning, particularly in tasks requiring integrated mathematical logic, spatial awareness, and creative insight.
Read more about our update here:
🔗 Blogpost → https://t.co/qLlzYSalUw
✨ Very overdue update:
I'll be starting as an Assistant Professor in CS at University of Minnesota, Twin Cities, Fall 2026. I will be recruiting PhD students!!
Please help me spread the word! [Thread] 1/n
PhD in Computer Science, University of California San Diego 🎓
My research focused on uncertainty and safety in AI systems, including
🤷♀️letting models say "I don't know" under uncertainty
🔎understanding and reducing hallucinations
🔁 methods for answering "how much will providing data X improve performance on Y?" at inference time
Many thanks to my advisor @berkustun, to my incredible research collaborators, and to my wonderful friends, husband and family. Getting a PhD while becoming a first-time parent is definitely a recipe for growth!
Our new ICML 2025 oral paper proposes a new unified theory of both Double Descent and Grokking, revealing that both of these deep learning phenomena can be understood as being caused by prime numbers in the network parameters 🤯🤯
🧵[1/8]
Explainable AI has long frustrated me by lacking a clear theory of what explanations should do. Improve use of a model for what? How? Given a task what's max effect explanation can have? It's complicated bc most methods are functions of features & prediction but not true state 1/
Explanations don't help us detect algorithmic discrimination. Even when users are trained. Even when we control their beliefs. Even under ideal conditions... 👇
Right to explanation laws assume explanations help people detect algorithmic discrimination.
But is there any evidence for that?
In our latest work w/ David Danks @berkustun, we show explanations fail to help people, even under optimal conditions.
PDF https://t.co/RlKmlzxgxN
We’ll be presenting @FAccTConference on 06.24 at 10:45 AM during the Evaluating Explainable AI session!
Come chat with us. We would love to discuss implications for AI policy, better auditing methods, and next steps for algorithmic fairness research.
#AIFairness#xAI
Right to explanation laws assume explanations help people detect algorithmic discrimination.
But is there any evidence for that?
In our latest work w/ David Danks @berkustun, we show explanations fail to help people, even under optimal conditions.
PDF https://t.co/RlKmlzxgxN
When RAG systems hallucinate, is the LLM misusing available information or is the retrieved context insufficient? In our #ICLR2025 paper, we introduce "sufficient context" to disentangle these failure modes. Work w J. Zhang, C.S. Ferng, @DaChengJuan1, @ankurtaly@CyrusRashtchian
Denied a loan, an interview, or an insurance claim by machine learning models? You may be entitled to a list of reasons.
In our latest w @anniewernerfelt@berkustun@kdphd, we show how existing explanation frameworks can fail and present an alternative tailored for recourse
🧵
Why is it so hard to show that people can be better decision-makers than statistical models? Some ways that common intuitions about the superiority of human judgment contradict statistical reality, and a few that don't.
https://t.co/5dgGMGuHXc
Many ML models predict labels that don’t reflect what we care about e.g.:
– Diagnoses from unreliable tests
– Outcomes from noisy electronic health records
In our #ICLR2025 paper, we study how this subjects individuals to a lottery of mistakes
Paper: https://t.co/TqQOMTwONy
🧵👇
🚨 Excited to announce a new paper accepted at ICLR2025 in Singapore!
“Learning Under Temporal Label Noise”
We tackle a new challenge in time series ML: label noise that changes over time 🧵👇
https://t.co/Ka7ABXArVR
🔥🔥Our small team in Seattle Google DeepMind is hiring! 🔥🔥If you are willing to move to/already in Seattle, has done significant work on human-machine communication / interpretability (from ML side) with a relevant PhD and great publication record, Join us.
Apply here 👉👉 https://t.co/HxU7zIcGMh
Are you interested in serving as a Program Committee member for the ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT 2025)? PC Members are expected to review papers in their area of expertise. Expression of interest form: https://t.co/RufvO5teu0
#FAccT2025