Ali Emami

Asst. Prof @ Emory CS | PhD McGill & Mila

[Ali Emami @_aliemami]

Honoured to receive the #EMNLP2024 Social Impact Paper Award with collaborators, led by Rob Morabito (at the time of submitting, an undergrad!)

When developed carefully, AI *can* be used for the betterment of society, & for learning a thing or two about ourselves along the way!

[Yiyou Sun @YiyouSun]

“AI agents will outperform humans at almost all jobs by 2026–2027.” - The forecast is everywhere.
So we built the exam to test that claim, on real labor-market aligned work. On the hardest tier, top agents pass 2.6%.
Meet Agents' Last Exam (ALE), a rolling benchmark measuring whether agents can actually do real jobs. 🧵👇

[Hassan Sajjad @hassaan84s]

@icmlconf Happy to share our work on Selective Conformal Optimized Pairwise Evaluation for LLM Judges

https://t.co/yI9loiM5m5

Phenomenal work by Sher, in collaboration with a great mentor @_aliemami

[Ali Emami @_aliemami]

@_joestacey_ @NafiseSadat @SheffieldNLP This is awesome, congrats Joe!!

[Ali Emami @_aliemami]

Beyond India, we lay out the complete pipeline for applying this to any culturally diverse country. Check out our paper: https://t.co/UAVlEWuPFe

Huge shoutout to co-authors Sangmitra Madhusudan, Trush Shashank, Steph Buongiorno, Renata Dividino & @jad_kabbara

[Ali Emami @_aliemami]

"Which decorations are typically used in the celebration of festivals?"

In North India, it's lights. In West India, flowers. In Central India..Rangolis!

Same question, same country, different answers.

LLMs treat Indian culture as monolithic. We built INDICA to test this!

[Ali Emami @_aliemami]

Cultural commonsense in India is regional, not national.

The LLMs we tested achieve only 13-20% accuracy on region-specific questions.

Worse, they consistently over-select answers from Central and Northern India while marginalizing other regions.

[Parisa Kordjamshidi @Kordjamshidi]

Our paper "FOREST: Frame of Reference Evaluation in Spatial Reasoning Tasks", won a Senior Area Chair Highlights Award at EMNLP-2025! Thanks to the excellent work of my graduate student Tanawan Premsri. #EMNLP2025 #NLProc #EMNLP https://t.co/VIvzl3K2Rp

[Ali Emami @_aliemami]

If you're at #EMNLP2025, we're presenting “We Politely Insist: Your LLM Must Learn the Persian Art of Taarof.”. Our presentation is also on Underline, & if you love it, let us know by casting your votes ❤️

🗓️ Nov 6, 8–9 AM (UTC+8)
📍 Gather Session 3
📰 https://t.co/FzHhLScYmY

[Fazl @ICML 🇰🇷 @FazlBarez]

We’re hiring!

Looking for Interns, Research Assistants, and Postdocs to work on Automated Interpretability--building systems that can analyse, explain, and intervene on large models to make them safe!

Work with me @Oxford, or remotely. Apply by Nov 15:
https://t.co/cL6dVOUcL2

I’ll be at #EMNLP this week--if you want to chat about automated interpretability, safety, technical governance etc, come say hi 👋

[Ali Emami @_aliemami]

Full paper (w/ reasoning traces & human comparison): https://t.co/4AlUHC2ncF

Huge shoutout to lead author Sangmitra Madhusudan and Kaige Chen (whose brilliance [that the models we tested repeatedly underestimated] powered this entire work).

[Ali Emami @_aliemami]

🚨 New paper: When do LLMs stop parsing language, and start guessing from semantic "reflexes"?

We built CenterBench, a stress test of recursive center-embedded sentences like:
“The cat [that the dog chased] meowed.”

[Ali Emami @_aliemami]

Interesting nugget for us🍗:

Plausibility actually hurts performance when the question requires causal reasoning, not world knowledge (pedagogical moral: shortcuts hinder you on tough questions!)

Models confidently hallucinate the likely consequence...not the grammatical one.

[Abhilasha Ravichander @lasha_nlp]

It is PhD application season again 🍂 For those looking to do a PhD in AI, these are some useful resources 🤖:

1. Examples of statements of purpose (SOPs) for computer science PhD programs: https://t.co/Stz53ZiREM [1/4]

[Cesare Spinoso-Di Piano @cesare_spinoso]

As one of the organizers of the @McGill_NLP reading group, I'm happy to announce that we've uploaded our very first talk to YouTube: https://t.co/arr8Qi6Ilm

[Ali Emami @_aliemami]

@universeinanegg It didn't manage to stop me from viewing and loving your work, Ari! Keep em coming!

[Ali Emami @_aliemami]

@mhashemi Taarof mogged is a hilarious way to put it 🤣