Assistant Professor, MIT EECS. Rigorous stats & ML methods for data-driven science and reliable AI systems.
My research group is hiring postdocs & PhDs!
📰 Excited to share our new work on risk control in prediction! Multiple testing leads to practical calibration algorithms with PAC guarantees for any statistical error rate. Works with any model + data distribution!
https://t.co/OCCQUdZsCi
#Statistics#MachineLearning
Thrilled to share Learn then Test, a tool to calibrate any model to control risk (eg. IOU, recall in object detection). No assns on model/data.
See arXiv https://t.co/kql7BfyMFb
+ Colab https://t.co/tZnpo2l6mn
✍️w/@stats_stephen, E.J. Candes, M.I. Jordan, @lihua_lei_stat! 🧵1/n
Excited that our ICML 2026 workshop Statistical Frameworks for Uncertainty in Agentic Systems got accepted 🎉 @icmlconf#icml2026
We want to bring together people thinking about uncertainty and agentic systems.
(1/5) Modern reasoning systems rely on test-time scaling: CoT, self-consistency, MCTS...
But two challenges remain:
1️⃣ Confidence signals shift across tasks/prompts
2️⃣ Stopping decisions are typically static and heuristic
We ask:
Can we adapt confidence within each reasoning trajectory — while still preserving statistical guarantees?
Calibrating LLM reasoning in test-time scaling is not new. But what if calibration itself could adapt online — at test time — to the specific reasoning trajectory of each instance?
Our new paper proposes a Test-Time Training framework for calibrating generalizable LLM reasoning, enabling instance-level adaptation with distribution-level robustness.
Paper: https://t.co/FtCD6gIZcN
Today I'm sharing a preprint on conformal risk control for non-monotonic losses, a paper three years in the making.
The key idea: validity of conformal can be reframed as a consequence of algorithmic stability. Therefore, any stable algorithm inherits a conformal guarantee.
🧵
we're hiring a Ph.D. intern! join us @genentech in South San Francisco for a summer advancing ML & statistical approaches for clinical trial design & analysis 📉💊DMs are open, feel free to reach out! 🔗https://t.co/4LRO9UkpnW
we're hiring a Ph.D. intern! join us @genentech in South San Francisco for a summer advancing ML & statistical approaches for clinical trial design & analysis 📉💊DMs are open, feel free to reach out! 🔗https://t.co/4LRO9UkpnW
I wrote a review paper about statistical methods in generative AI; specifically, about using statistical tools along with genAI models for making AI more reliable, for evaluation, etc. See here: https://t.co/0aq8hJqXzo!
I have identified four main areas where statistical thinking can be helpful. These are just a subset of what is out there; other topics have been well-covered in other reviews.
1. Designing "statistical wrappers" around a model, for instance, changing behavior of a trained model (e.g., abstaining), where a score, e.g., an "unsafety score" is too high. The key connection to statistics is to use the quantiles of the loss (on a calibration set) to set the critical threshold, thus enabling conformal-type high probability guarantees.
2. Closely related, methods for uncertainty quantification, which enable the model to express uncertainty in an answer. A crucial component here is "calibration", whereby the uncertainty is required to reflect reality.
3. Statistical methods for AI evaluation: Specifically, tools for statistical inference (e.g., confidence intervals) on model performance. Exciting recent work proposes careful statistical models for leveraging a very small high-quality dataset, possibly combined with much larger low-quality datasets, for accurate evaluation.
4. Experiment design and interventions. Careful AI experiments to understand and steer models may require interventions such as modifying experimental settings in a controlled manner. This brings up connections to classical experimental design in statistics. This connection has largely remained implicit so far, and my review aims to make it more explicit; hoping that experimental design principles will become useful here.
This review references the work of many, including @HamedSHassani@obastani@tatsu_hashimoto @yuekai_sun @CsabaSzepesvari@ml_angelopoulos@stats_stephen@yaniv_romano@yaringal@KilianQW@_onionesque +their teams, and some work that I was also involved in.
Hopefully, my review will be helpful to orient yourself in this exciting area. Nonetheless, since the area is rapidly expanding, it is possible that I missed important references. Please feel free to let me know of anything that I should add/change!
If you work at the intersection of CS and economics (or think your work is of interest to those who do!) consider submitting to the ESIF Economics and AI+ML meeting this summer at Cornell: https://t.co/ZpCrofc8Fn
(1/5) Beyond Next-Token Prediction, introducing Next Semantic Scale Prediction! Our @NeurIPSConf NeurIPS 2025 paper HDLM is out! Check out the new language modeling paradigm: Next Semantic Scale Prediction via Hierarchical Diffusion Language Models.
It largely generalizes Masked Diffusion Models (MDM), and provides the progressively denoising capability for each token in the semantic level. Minimal computation overheads, much better results!
arxiv: https://t.co/CwGqnUptzX
code: https://t.co/asiDuxKw8w
Happy to share that our paper on how to obtain reliable statistical inferences from satellite-based maps is now published in Remote Sensing of Environment!
Today, NSF announced an add’l 500 NSF Graduate Research Fellowship Program awardees for the 2025-2026 cohort, bringing the total to approx 1,500. #NSFGRFP supports grad students as they pursue their dreams, build STEM skills, & become the next generation of innovators & leaders.
📢If you're interested in conformal prediction, algorithms w/predictions, robust stats & connections between them from a theory perspective, join us for a workshop at #COLT2025 in Lyon 🇫🇷 June 30!
Submit a poster description by May 25, more here:
https://t.co/gXa88zx53F
Our paper notifications are out! Congratulations to the authors and look forward to an exciting lineup of discussions.
Stay tuned for more details! #ICLR2025
🙌🎉Our 2025 recipient of the COPSS Presidents' Award, is Lester Mackey! This award is given annually to a young member of the statistical community in recognition of outstanding contributions to the profession of statistics.
📢 We are hiring a postdoc to work on remote sensing of soil carbon and land degradation! 🌱🗺️ The position will be hosted by the Earth Intelligence Lab & @mitenergy, with an earliest start date of April 2025.
To apply: https://t.co/Hx0U91DL3x