The Approximately Correct Machine Intelligence (ACMI) Lab at @mldcmu at @SCSatCMU. Growing the ML sandbox to address more of the real world. PI @zacharylipton
Yesterday, I had the honor of speaking in the US Senate AI Insight Forum on privacy & liability, moderated by @SenSchumer@SenatorRounds@SenatorHeinrich & @SenToddYoung. Each panelist submitted a written statement for @SenSchumer's website. Here’s mine:
https://t.co/KEp7eygTAB
New paper "Domain Adaptation under Open Set Label Shift" by @acmi_lab PhD student Saurabh Garg w coadvisors @zacharylipton & Siva Balakrishnan.
Lays out theoretical foundations & practical algorithm, for one scenario where open set adaptation can work.
https://t.co/iH92s4ZllA
Excited to share new @acmi_lab paper introducing the first(?) theoretically coherent setting for open set classification. Under the label shift assumption, we can now handle both label shift (among prev seen classes) & arrival of a never-before-seen class
https://t.co/jrxxWMQmz3
Is flatness indicative of generalization? Not necessarily.
Our experimental study calls the relationship between flatness (as measured by the max Hessian eigenvalue) and generalization into question.
https://t.co/ORln4ASVEq
New work by @acmi_lab PhD student @dkaushik96 tackles the thorny issue of when to designate ML crowdworkers as human subjects. Our analysis reveals nuances, ambiguities, a loophole, & practical guidance
authors: DK, @zacharylipton & @AlexJohnLondon
paper: https://t.co/CFlTvSPF5b
Preprint alert 🚨
With ML’s growing reliance on crowdsourcing, in this paper, @zacharylipton, @AlexJohnLondon, and I seek to resolve the human subject status of ML’s crowdworkers. More in the thread🧵 1/15
https://t.co/iBuggj4OaB
Congrats to our nephew-turned-son Riccardo Fogliato on a great thesis proposal. Riccardo's tackles deep questions re (i) the performance and fairness properties of criminal risk assessment instruments; and (ii) {human+model} hybrid decision-making systems.
https://t.co/TEolMJWPUJ
Congratulations to @CMU_Stats Riccardo Fogliato on his successful PhD thesis proposal on “Data and Humans in Algorithmic Risk Assessment”!! Co-advised by Alexandra Chouldechova @HeinzCollege and Zachary Lipton @zacharylipton@mldcmu@teppercmu
New work by @saurabh_garg67 (ICLR 2022) shows that in general, OOD accuracy is identified only when the optimal predictor is identified. Thus, any guarantee requires assumptions on nature of shift. Also discovers a simple method that works surprisingly well on many benchmarks.
"Can we predict OOD performance given access to unlabeled target data?"
We investigate methods to predict target domain performance and find a simple method that does surprisingly well.
Paper: https://t.co/2P7QQl4vR2
with Siva B, @zacharylipton, @bneyshabur, @HanieSedghi
1/
Congrats to our 2nd ever PhD, the soon-to-be-minted Doctor Danish, who defended his dissertation this week. Danish joined this lab before it was a lab and helped build it from the ground up. We're proud of all you've accomplished and excited to see your future unfold. 👨🎓📜💻
Excited to end the year on a high: I passed my PhD defense today!
*Absolutely* loved my PhD years @LTIatCMU—I could have spent another 3 years! Major thanks to @zacharylipton, @gneubig, @professorwcohen for being wonderful advisors. [1/n]
@postmachines Eventually, they do. But potential is not the same thing as mastery. Scientific values and practice are propagated through an apprenticeship model. They come to learn the craft.
Research experience is great, published papers can be impressive, but (generally) they are neither necessary nor sufficient for joining our lab. Some of our criteria:
@kuchhal_dhruv@mynkgoel Great Q & there's too much to cover in a tweet (e.g. there's getting past departmental admit processes vs our lab's process). Some things that help: written statements (we really read them), interviews (both PI & students), record of projects coming alive (even outside research)
Pubs on a CV signal many things. It's not easy to bang out papers pre-PhD. But merely knowing that a researcher has been published or even that they have reliably have contributed to projects that met the bar for conference peer review carries little signal re the above.
4. Fire—will this person bring some attitude to the lab? Can they forcefully disagree when appropriate? Will they spot flaws in a research direction? can can they cut against consensus? Will they spark creative directions, and do they have the drive to push them into reality? 🔥
In addition to modeling satiation, our key technical innovation—modeling rewards as dynamical systems—may have broader applications & (given a different parameterization) be used to model other phenomena, such as brand loyalty & binging, & may prove useful beyond RecSys. (6/n)