Clemencia (Clem)

Postdoc @CWI, PhD @irlab_amsterdam @UvA_Amsterdam| Conversational Search Systems & Evaluation with users | Commonly known as Clem, @MasakhaneNLP

[Clemencia (Clem) @cnsiro]

On July 8th, I successfully defended my PhD 🎓✨
My thesis, "Rethinking the Human-Centered Evaluation of Conversational Systems," explores aligning conversation evaluation with real human needs.
Thanks to my supervisors @mdr and @maliannejadi for their support.

[Clemencia (Clem) @cnsiro]

Joint work with Pourya Allianejadi and @maliannejadi

[Clemencia (Clem) @cnsiro]

𝗟𝗟𝗠𝘀 𝗱𝗼𝗻'𝘁 𝗷𝘂𝘀𝘁 𝗲𝘃𝗮𝗹𝘂𝗮𝘁𝗲.
𝗧𝗵𝗲𝘆 𝗱𝗲𝗰𝗶𝗱𝗲 𝘄𝗵𝗮𝘁 "𝗴𝗼𝗼𝗱" 𝗺𝗲𝗮𝗻𝘀.

In 𝘓𝘦𝘢𝘳𝘯𝘪𝘯𝘨 𝘵𝘰 𝘑𝘶𝘥𝘨𝘦 (EACL 2026), we test this:

→ Let LLMs write their own evaluation rules
→ Then apply them

[Clemencia (Clem) @cnsiro]

𝙀𝙖𝙘𝙝 𝙢𝙤𝙙𝙚𝙡 𝙙𝙚𝙫𝙚𝙡𝙤𝙥𝙨 𝙞𝙩𝙨 𝙤𝙬𝙣 𝙚𝙫𝙖𝙡𝙪𝙖𝙩𝙞𝙤𝙣 𝙙𝙞𝙖𝙡𝙚𝙘𝙩.

𝘓𝘓𝘔-𝘣𝘢𝘴𝘦𝘥 𝘦𝘷𝘢𝘭𝘶𝘢𝘵𝘪𝘰𝘯 𝘪𝘴 𝘯𝘰𝘵 𝘯𝘦𝘶𝘵𝘳𝘢𝘭. 𝘐𝘵 𝘳𝘦𝘧𝘭𝘦𝘤𝘵𝘴 𝘩𝘰𝘸 𝘮𝘰𝘥𝘦𝘭𝘴 𝘥𝘦𝘧𝘪𝘯𝘦 𝘲𝘶𝘢𝘭𝘪𝘵𝘺.

📄 https://t.co/55tuivQrNo

[AfricaNLP @Africa_NLP]

🚨 AfricaNLP 2026 First Call for Paper

The 7th AfricaNLP Workshop at EACL 2026 (Rabat, Morocco) invites papers on Multilingual Multimodal LLMs for African languages.

👉 Submission deadline: December 19 2025 (AoE)
Details: https://t.co/aBDcSRcJxJ

#AfricaNLP #EACL2026 #NLP

[PALS NLP Workshop @pals_nlp_wrkshp]

We’re now accepting ARR commitments for PALS 2025! If your ARR-reviewed paper fits with our themes on LLM personalization, submit by September 4: https://t.co/dm35LywSVh

See the full call for papers and topic details on our website: https://t.co/LV6DkyHuY0
#EMNLP25 @emnlpmeeting

[PALS NLP Workshop @pals_nlp_wrkshp]

3 days remaining for direct submissions to PALS 2025! Share your findings or works in progress on LLM personalization here: https://t.co/xNb5BH8cod…

See our website for the call for papers and information about relevant topics: https://t.co/t9igCuB05i

#EMNLP25 @emnlpmeeting

[LLM4Eval Workshop @llm4eval]

You are welcome to join us at Giotto Platea for the LLM4Eval workshop #SIGIR2025 .

[LLM4Eval Workshop @llm4eval]

Chirag Shah is this morning giving a keynote on: From Relevance to Reality: Scaling Human-Centered Evaluation in the LLM Era

[Clemencia (Clem) @cnsiro]

Grateful to my committee: Evangelos Kanoulas, @DrCh0le , @sivareddyg , @zhaochun_ren , @theoaraujo , @HelbergerN7484, Fabio Crestani, and @JudithGood (chair).

The thesis is available online: https://t.co/qPDRwCN48j

[Clemencia (Clem) @cnsiro]

On July 8th, I successfully defended my PhD 🎓✨
My thesis, "Rethinking the Human-Centered Evaluation of Conversational Systems," explores aligning conversation evaluation with real human needs.
Thanks to my supervisors @mdr and @maliannejadi for their support.

[PALS NLP Workshop @pals_nlp_wrkshp]

Join us at @emnlpmeeting for:

"Tailoring AI: Exploring Active and Passive LLM Personalization" 🎯🧠

To answer, when should LLMs personalize? What role do users play in LLM-personalization?

📅 Deadline Aug. 1
📝 Details in thread 🧵👇
#EMNLP2025 #LLM #AI #personalization

1/5

[LLM4Eval Workshop @llm4eval]

📢 Deadline extended!
Submit to #LLM4Eval @ #SIGIR2025 by April 30 (AoE)

We welcome work on LLM-based evaluation, including:
🧠 beyond query-doc
🗣️ simulating users
🧪 synthetic eval data

🔗 https://t.co/B88X3U5wWX

#CFP #IR #LLMs #SIGIR2025

[David Ifeoluwa Adelani 🇳🇬 @davlanade]

If you are have submitted a paper to ARR/ACL evaluating on African languages and not happy with the meta reviews or unsure if it will get into ACL 2025

You can consider submitting your paper and the reviews to our AfricaNLP workshop https://t.co/4tCUfCbdRR

[Clemencia (Clem) @cnsiro]

cc @srahmanidashti, @maliannejadi, @nickcr, @claclarke, @guglielm0f, @UnderdogGeek, @pt_ir, and @emine_yilm

#SIGIR #IR #Evaluation #LLM4Eval #NLP

[Clemencia (Clem) @cnsiro]

Glad to be organizing the third series of #LLM4Eval @ #SIGIR2025!

We have just announced three special themes we are especially excited about: beyond query doc evaluation, simulating users, and synthesizing evaluation data.

Submit by April 23 🧠🗣️🧪

[LLM4Eval Workshop @llm4eval]

LLMs are reshaping how we evaluate ranked lists, conversations, and even simulate users.

At #LLM4Eval @ #SIGIR2025, we’re spotlighting 3 special themes:
🧠 Beyond query-doc
🗣️ Simulating users
🧪 Synthesizing evaluation data
👇 Curious? Read on & submit by April 23!
#LLMs #IR

[LLM4Eval Workshop @llm4eval]

📢 Call for Papers: LLM4EVAL @ SIGIR '25 - The 3rd Workshop on Large Language Models for Evaluation in IR! @SIGIRConf

🗓️ Submission: April 23, 2025 (AoE)
🔗 Full CFP: https://t.co/B88X3U5wWX
🌐 Workshop: https://t.co/mc84kebwB0

[Siva Reddy @sivareddyg]

I have multiple vacancies for PhD and Masters students at @Mila_Quebec @McGill_NLP in NLP/ML focusing on representation learning, reasoning, multimodality and alignment. Deadline for applications is Dec 1st. More details: https://t.co/MDofkzwpag

[Clemencia (Clem) @cnsiro]

- 👥🤖 A crowd of LLMs are better evaluators than a single LLM instance.
- ✅🤝CrowdLLM evaluations align with human evaluations
- LLM questions outperform human-written ones in several multidimensional aspects and retrieval performance 📈
🧵(3/3)

[Clemencia (Clem) @cnsiro]

New paper! "AGENT-CQ: Automatic Generation and Evaluation of Clarifying Questions for Conversational Search with LLMs" with @yfyuan775, @maliannejadi, and @mdr is now available at https://t.co/ddJGK4Rcb8 🚨

AGENT-CQ is an end-to-end LLM-based framework!

🧵(1/3)

[Clemencia (Clem) @cnsiro]

The framework has two parts:

1. Generation: Generates diverse clarifying questions using different prompting strategies and simulates user answers.
2. Evaluation (CrowdLLM): simulates human crowdsourcing judgments using multiple LLM instances to assess quality.
🧵(2/3)