It’s deployment time!
You’ve done the pre-deployment evals. You THINK your model is safe, so you ship it 🚀
🚨 After deployment, reports of misbehavior start trickling in
What happened?? How could you have caught it?? 🤔
@icmlconf 2026 Spotlight!
🧵
@seohong_park Yeah! We wondered the same in casual inference / offline RL.
We found earliest disagreement times, a type of adaptive time scale, are a useful concept for continuous or finely discretized times.
Could benefit from some large scale experiments, lots to do!
https://t.co/uWC2wwG8Sx
Life update: I'm starting as faculty at Boston University in 2026! BU has SCHEMES for LM interpretability & analysis, so I couldn't be more pumped to join a burgeoning supergroup w/ @najoungkim@amuuueller. Looking for my first students, so apply and reach out!
What prompt generated the image on the right?
Come find out today at our tutorial on OOD generalization: Shortcuts, Spuriousness, and Stability
@Maggiemakar @aahladpuli
Panel: @ElanRosenfeld@AdtRaghunathan Danica Sutherland
I'm recruiting PhD students for Fall 2025! CS PhD Deadline: Dec. 15th.
I work on safe/reliable ML and causal inference, motivated by healthcare applications.
Beyond myself, Johns Hopkins has a rich community of folks doing similar work! Come join us!
Unpacking what's going on in our paper on Med-* foundation models & their failure to improve over their generic counterparts (think Med-{LLaMa, LLaVa} vs {LLaMa, LLaVa}. A familiar tale of motivated reasoning, sloppy eval, & hidden hyper-optimization.
https://t.co/HDnJVxiHcO
I'm recruiting PhD students this cycle! My lab works at the intersection of information theory, cognition, language, and AI. Wanna hear more? I asked notebookLM to generate a podcast just for you (but pls take it with a big grain of salt...)
https://t.co/YGPrJEh8gm
Multimodal representation learning works for 2 modalities, but what if you're working with 3+ modalities, like in healthcare, robotics, or video?
📢 Meet Symile: a model-agnostic contrastive loss for any number of modalities with CLIP's simplicity and superior performance✨
1/n
Will your LLM model generalize over time or across sites? How do we improve robustness? This is a key issue LLMs struggle with and especially critical as we apply these models to healthcare applications.
👇 New clever technique to improve LLM robustness and generalization. Published @NeurIPSConf w/ wonderful postdoctoral fellow @wald_yoav & superb co-authors from Columbia.
https://t.co/1Am9t3yBaQ
https://t.co/iDg68Lnn10
Drop by our #NeurIPS2023 poster (#910) today (Wed.) at 5PM!
We improve OOD generalization in text applications with Language Models + Causal-inference-inspired data augmentation.
w/ best possible co-lead @AmirFeder , and @causalclaudia, @suchisaria, David Blei
Also, better late than never, yesterday @aahladpuli presented his cool work about shortcut learning in perception tasks. Some nice insights about useful inductive biases in these tasks.
https://t.co/Ajaq1JT0Bg
Thank you @ERC_Research for this amazing grant- I will
work hard (with and without the algorithms’ help) on these topics for the next few years and would love to update you with exciting results soon.
Once more, in verbose mode: as of today I am an Assistant Professor @WaterlooENG (ECE dept), and faculty affiliate @VectorInst and @TorontoSRI.
Please help me get the word out - I will be hiring motivated students who want to study machine learning and its societal implications
This comes up in monitoring safety, where novel types of data necessitates a reevaluation of existing models.
Under a "mild" assumption, we give a method + bounds on detecting the novel subgroup.
Our #UAI2023 paper is on detecting novel subgroups (aka classes/categories) when distribution of non-novel data shifts.
If this sounds interesting, or you’re wondering what it has to do with unicorns, check out our poster/paper/drop a line.
w/ @suchisaria
https://t.co/7nrMfDblfx
Excited to be moderating panel on generalization, scaling and safety at #icml2023 with great panelists @sleepinyourhat@zacharylipton and @Maggiemakar. Look forward to seeing folks at 15:30 room 316 at Workshop on Spurious Correlations, Invariance and Stability.