Stephen Bates

Assistant Professor, MIT EECS. Rigorous stats & ML methods for data-driven science and reliable AI systems.

My research group is hiring postdocs & PhDs!

[Stephen Bates @stats_stephen]

📰 Excited to share our new work on risk control in prediction! Multiple testing leads to practical calibration algorithms with PAC guarantees for any statistical error rate. Works with any model + data distribution!

https://t.co/OCCQUdZsCi

#Statistics #MachineLearning

[Anastasios Nikolas Angelopoulos @ml_angelopoulos]

Thrilled to share Learn then Test, a tool to calibrate any model to control risk (eg. IOU, recall in object detection). No assns on model/data.

See arXiv https://t.co/kql7BfyMFb
+ Colab https://t.co/tZnpo2l6mn

✍️w/@stats_stephen, E.J. Candes, M.I. Jordan, @lihua_lei_stat! 🧵1/n

[Stephen Bates @stats_stephen]

Announcing the Statistical Frameworks for Uncertainty in Agentic Systems workshop at ICML '26!

[Mahmoud Hegazy @oumatheu]

Excited that our ICML 2026 workshop Statistical Frameworks for Uncertainty in Agentic Systems got accepted 🎉 @icmlconf #icml2026

We want to bring together people thinking about uncertainty and agentic systems.

[Cai Zhou @zhuci19]

(1/5) Modern reasoning systems rely on test-time scaling: CoT, self-consistency, MCTS...

But two challenges remain:
1️⃣ Confidence signals shift across tasks/prompts
2️⃣ Stopping decisions are typically static and heuristic

We ask:
Can we adapt confidence within each reasoning trajectory — while still preserving statistical guarantees?

Calibrating LLM reasoning in test-time scaling is not new. But what if calibration itself could adapt online — at test time — to the specific reasoning trajectory of each instance?

Our new paper proposes a Test-Time Training framework for calibrating generalizable LLM reasoning, enabling instance-level adaptation with distribution-level robustness.

Paper: https://t.co/FtCD6gIZcN

[Anastasios Nikolas Angelopoulos @ml_angelopoulos]

Today I'm sharing a preprint on conformal risk control for non-monotonic losses, a paper three years in the making.

The key idea: validity of conformal can be reframed as a consequence of algorithmic stability. Therefore, any stable algorithm inherits a conformal guarantee.

🧵

[Stephen Bates @stats_stephen]

Postdoc opportunity — If you do ML/stat/applied math/… and want to work at the frontier of biology , come join us! 🤖 🧬

[Eric and Wendy Schmidt Center @Schmidt_Center]

Interested in pursuing #machinelearning, #appliedmathematics, #statistics, or #computationalresearch to work on biomedical problems at the @broadinstitute? Apply to become a @Schmidt_Center postdoctoral associate: https://t.co/VXqP62c19w

[Eric and Wendy Schmidt Center @Schmidt_Center]

Interested in pursuing #machinelearning, #appliedmathematics, #statistics, or #computationalresearch to work on biomedical problems at the @broadinstitute? Apply to become a @Schmidt_Center postdoctoral associate: https://t.co/VXqP62c19w

[Eric and Wendy Schmidt Center @Schmidt_Center]

🎉 Our new machine learning challenge – Obesity ML Competition: Tackling Metabolic Diseases – is officially open! Register, watch our introduction videos and lecture series, and begin coding today: https://t.co/h59QRwlJdG

@broadinstitute @crunchDAO

[Stephen Bates @stats_stephen]

Exciting research internship!

[Clara Fannjiang @clara_fannjiang]

we're hiring a Ph.D. intern! join us @genentech in South San Francisco for a summer advancing ML & statistical approaches for clinical trial design & analysis 📉💊DMs are open, feel free to reach out! 🔗https://t.co/4LRO9UkpnW

[Clara Fannjiang @clara_fannjiang]

we're hiring a Ph.D. intern! join us @genentech in South San Francisco for a summer advancing ML & statistical approaches for clinical trial design & analysis 📉💊DMs are open, feel free to reach out! 🔗https://t.co/4LRO9UkpnW

[Edgar Dobriban @EdgarDobriban]

I wrote a review paper about statistical methods in generative AI; specifically, about using statistical tools along with genAI models for making AI more reliable, for evaluation, etc. See here: https://t.co/0aq8hJqXzo!

I have identified four main areas where statistical thinking can be helpful. These are just a subset of what is out there; other topics have been well-covered in other reviews.

1. Designing "statistical wrappers" around a model, for instance, changing behavior of a trained model (e.g., abstaining), where a score, e.g., an "unsafety score" is too high. The key connection to statistics is to use the quantiles of the loss (on a calibration set) to set the critical threshold, thus enabling conformal-type high probability guarantees.

2. Closely related, methods for uncertainty quantification, which enable the model to express uncertainty in an answer. A crucial component here is "calibration", whereby the uncertainty is required to reflect reality.

3. Statistical methods for AI evaluation: Specifically, tools for statistical inference (e.g., confidence intervals) on model performance. Exciting recent work proposes careful statistical models for leveraging a very small high-quality dataset, possibly combined with much larger low-quality datasets, for accurate evaluation.

4. Experiment design and interventions. Careful AI experiments to understand and steer models may require interventions such as modifying experimental settings in a controlled manner. This brings up connections to classical experimental design in statistics. This connection has largely remained implicit so far, and my review aims to make it more explicit; hoping that experimental design principles will become useful here.

This review references the work of many, including @HamedSHassani @obastani @tatsu_hashimoto @yuekai_sun @CsabaSzepesvari @ml_angelopoulos @stats_stephen @yaniv_romano @yaringal @KilianQW @_onionesque +their teams, and some work that I was also involved in.

Hopefully, my review will be helpful to orient yourself in this exciting area. Nonetheless, since the area is rapidly expanding, it is possible that I missed important references. Please feel free to let me know of anything that I should add/change!

[Aaron Roth @Aaroth]

If you work at the intersection of CS and economics (or think your work is of interest to those who do!) consider submitting to the ESIF Economics and AI+ML meeting this summer at Cornell: https://t.co/ZpCrofc8Fn

[Cai Zhou @zhuci19]

(1/5) Beyond Next-Token Prediction, introducing Next Semantic Scale Prediction! Our @NeurIPSConf NeurIPS 2025 paper HDLM is out! Check out the new language modeling paradigm: Next Semantic Scale Prediction via Hierarchical Diffusion Language Models.

It largely generalizes Masked Diffusion Models (MDM), and provides the progressively denoising capability for each token in the semantic level. Minimal computation overheads, much better results!

arxiv: https://t.co/CwGqnUptzX

code: https://t.co/asiDuxKw8w

[Sherrie Wang @sherwang]

Happy to share that our paper on how to obtain reliable statistical inferences from satellite-based maps is now published in Remote Sensing of Environment!

[U.S. National Science Foundation @NSF]

Today, NSF announced an add’l 500 NSF Graduate Research Fellowship Program awardees for the 2025-2026 cohort, bringing the total to approx 1,500. #NSFGRFP supports grad students as they pursue their dreams, build STEM skills, & become the next generation of innovators & leaders.

[Jessica Hullman @JessicaHullman]

📢If you're interested in conformal prediction, algorithms w/predictions, robust stats & connections between them from a theory perspective, join us for a workshop at #COLT2025 in Lyon 🇫🇷 June 30!

Submit a poster description by May 25, more here:
https://t.co/gXa88zx53F

[Massachusetts Institute of Technology (MIT) @MIT]

Imagine a world without MIT.

[Stephen Bates @stats_stephen]

@Micro_Yunha @MIT @MITBiology @MITEECS @MIT_SCC Welcome Yunha!

[Sharon Li @SharonYixuanLi]

Our paper notifications are out! Congratulations to the authors and look forward to an exciting lineup of discussions.

Stay tuned for more details! #ICLR2025

[COPSS @COPSSNews]

🙌🎉Our 2025 recipient of the COPSS Presidents' Award, is Lester Mackey! This award is given annually to a young member of the statistical community in recognition of outstanding contributions to the profession of statistics.

[Stephen Bates @stats_stephen]

@COPSSNews @LesterMackey Congratulations @LesterMackey! Wonderful news :)

[Sherrie Wang @sherwang]

📢 We are hiring a postdoc to work on remote sensing of soil carbon and land degradation! 🌱🗺️ The position will be hosted by the Earth Intelligence Lab & @mitenergy, with an earliest start date of April 2025.

To apply: https://t.co/Hx0U91DL3x