ACMI Lab (CMU)

The Approximately Correct Machine Intelligence (ACMI) Lab at @mldcmu at @SCSatCMU. Growing the ML sandbox to address more of the real world. PI @zacharylipton

[ACMI Lab (CMU) @acmi_lab]

RecSys often assumes static rewards but preferences evolve! Consider “satiation”: 🍕for meal 1: 😄, 🍕for meal 2: 🤔, 🍕for meal 3: 😭… [no🍕] … 🍕for meal 100: 😄.

In “Rebounding Bandits” we model dynamic rewards w linear dynamical systems https://t.co/Fm2EVucoSS #neurips2021

[Zachary Lipton @zacharylipton]

Yesterday, I had the honor of speaking in the US Senate AI Insight Forum on privacy & liability, moderated by @SenSchumer @SenatorRounds @SenatorHeinrich & @SenToddYoung. Each panelist submitted a written statement for @SenSchumer's website. Here’s mine:
https://t.co/KEp7eygTAB

[Divyansh Kaushik @dkaushik96]

An @acmi_lab reunion at @icmlconf! #ICML2023

[ACMI Lab (CMU) @acmi_lab]

New paper "Domain Adaptation under Open Set Label Shift" by @acmi_lab PhD student Saurabh Garg w coadvisors @zacharylipton & Siva Balakrishnan.

Lays out theoretical foundations & practical algorithm, for one scenario where open set adaptation can work.

https://t.co/iH92s4ZllA

[Zachary Lipton @zacharylipton]

Excited to share new @acmi_lab paper introducing the first(?) theoretically coherent setting for open set classification. Under the label shift assumption, we can now handle both label shift (among prev seen classes) & arrival of a never-before-seen class
https://t.co/jrxxWMQmz3

[Simran Kaur @kaur_simran25]

Is flatness indicative of generalization? Not necessarily.

Our experimental study calls the relationship between flatness (as measured by the max Hessian eigenvalue) and generalization into question.

https://t.co/ORln4ASVEq

[ACMI Lab (CMU) @acmi_lab]

New work by @acmi_lab PhD student @dkaushik96 tackles the thorny issue of when to designate ML crowdworkers as human subjects. Our analysis reveals nuances, ambiguities, a loophole, & practical guidance

authors: DK, @zacharylipton & @AlexJohnLondon
paper: https://t.co/CFlTvSPF5b

[Divyansh Kaushik @dkaushik96]

Preprint alert 🚨

With ML’s growing reliance on crowdsourcing, in this paper, @zacharylipton, @AlexJohnLondon, and I seek to resolve the human subject status of ML’s crowdworkers. More in the thread🧵 1/15
https://t.co/iBuggj4OaB

[ACMI Lab (CMU) @acmi_lab]

Congrats to our nephew-turned-son Riccardo Fogliato on a great thesis proposal. Riccardo's tackles deep questions re (i) the performance and fairness properties of criminal risk assessment instruments; and (ii) {human+model} hybrid decision-making systems.
https://t.co/TEolMJWPUJ

[Carnegie Mellon Statistics & Data Science @CMU_StatDS]

Congratulations to @CMU_Stats Riccardo Fogliato on his successful PhD thesis proposal on “Data and Humans in Algorithmic Risk Assessment”!! Co-advised by Alexandra Chouldechova @HeinzCollege and Zachary Lipton @zacharylipton @mldcmu @teppercmu

[ACMI Lab (CMU) @acmi_lab]

New work by @saurabh_garg67 (ICLR 2022) shows that in general, OOD accuracy is identified only when the optimal predictor is identified. Thus, any guarantee requires assumptions on nature of shift. Also discovers a simple method that works surprisingly well on many benchmarks.

[Saurabh Garg @saurabh_garg67]

"Can we predict OOD performance given access to unlabeled target data?"

We investigate methods to predict target domain performance and find a simple method that does surprisingly well.

Paper: https://t.co/2P7QQl4vR2

with Siva B, @zacharylipton, @bneyshabur, @HanieSedghi

1/

[ACMI Lab (CMU) @acmi_lab]

Congrats to our 2nd ever PhD, the soon-to-be-minted Doctor Danish, who defended his dissertation this week. Danish joined this lab before it was a lab and helped build it from the ground up. We're proud of all you've accomplished and excited to see your future unfold. 👨‍🎓📜💻

[Danish Pruthi @danish037]

Excited to end the year on a high: I passed my PhD defense today!

*Absolutely* loved my PhD years @LTIatCMU—I could have spent another 3 years! Major thanks to @zacharylipton, @gneubig, @professorwcohen for being wonderful advisors. [1/n]

[ACMI Lab (CMU) @acmi_lab]

@postmachines Eventually, they do. But potential is not the same thing as mastery. Scientific values and practice are propagated through an apprenticeship model. They come to learn the craft.

[ACMI Lab (CMU) @acmi_lab]

Research experience is great, published papers can be impressive, but (generally) they are neither necessary nor sufficient for joining our lab. Some of our criteria:

[ACMI Lab (CMU) @acmi_lab]

@kuchhal_dhruv @mynkgoel Great Q & there's too much to cover in a tweet (e.g. there's getting past departmental admit processes vs our lab's process). Some things that help: written statements (we really read them), interviews (both PI & students), record of projects coming alive (even outside research)

[ACMI Lab (CMU) @acmi_lab]

Pubs on a CV signal many things. It's not easy to bang out papers pre-PhD. But merely knowing that a researcher has been published or even that they have reliably have contributed to projects that met the bar for conference peer review carries little signal re the above.

[ACMI Lab (CMU) @acmi_lab]

4. Fire—will this person bring some attitude to the lab? Can they forcefully disagree when appropriate? Will they spot flaws in a research direction? can can they cut against consensus? Will they spark creative directions, and do they have the drive to push them into reality? 🔥

[ACMI Lab (CMU) @acmi_lab]

Work led by ACMI PhD student @leqi_liu, with Fatma Kılınç-Karzan, @zacharylipton, and Alan Montgomery.

Paper link: https://t.co/Fm2EVucoSS (n=7/n)

[ACMI Lab (CMU) @acmi_lab]

RecSys often assumes static rewards but preferences evolve! Consider “satiation”: 🍕for meal 1: 😄, 🍕for meal 2: 🤔, 🍕for meal 3: 😭… [no🍕] … 🍕for meal 100: 😄.

In “Rebounding Bandits” we model dynamic rewards w linear dynamical systems https://t.co/Fm2EVucoSS #neurips2021

[ACMI Lab (CMU) @acmi_lab]

In addition to modeling satiation, our key technical innovation—modeling rewards as dynamical systems—may have broader applications & (given a different parameterization) be used to model other phenomena, such as brand loyalty & binging, & may prove useful beyond RecSys. (6/n)