LLM Evals Workshop @NeurIPS

NeurIPS 2025 Workshop. Evaluating the Evolving LLM Lifecycle: Benchmarks, Emergent Abilities, and Scaling

[LLM Evals Workshop @NeurIPS @LLM_eval]

We are happy to announce our @NeurIPSConf workshop on LLM evaluations!

Mastering LLM evaluation is no longer optional -- it's fundamental to building reliable models. We'll tackle the field's most pressing evaluation challenges.

For details: https://t.co/Rithk3osFH. 1/3

[Berivan Isik @BerivanISIK]

It has been a super fun day @LLM_eval workshop @NeurIPSConf with amazing talks, posters, and an engaging panel discussion!

@dawnsongtweets @natolambert @orf_bnw @sanmikoyejo @abeirami @hamishivi @MariusHobbhahn @beyzaermis @Diyi_Yang @attaluri_nithya @RishiBommasani @YangjunR

[Berivan Isik @BerivanISIK]

Our next talk @LLM_eval workshop is by @sanmikoyejo!

Upper Level Room 2 @NeurIPSConf

[Berivan Isik @BerivanISIK]

“Good researchers obsess over evals” by @natolambert @LLM_eval workshop!

[Nithya Attaluri @attaluri_nithya]

Bringing the hot take culture to NeurIPS - great talk @orf_bnw!!

[Berivan Isik @BerivanISIK]

@dawnsongtweets is giving a talk on agentic evals @LLM_eval workshop!

[Berivan Isik @BerivanISIK]

@LLM_eval workshop has started with Orhan Firat’s talk at Upper Level Room 2.

@NeurIPSConf

[Nathan Lambert @natolambert]

Good researchers obsess over evals
The story of Olmo 3 (post-training), told through evals
NeurIPS Talk tomorrow.
Upper Level Room 2, 10:35AM.

[Berivan Isik @BerivanISIK]

I’ll be @NeurIPSConf all week and would love to connect on LLM data, evaluation, benchmarking, and scaling laws. If you’re working on related problems, feel free to reach out.

PS: Don’t miss our one-of-a-kind workshop on LLM evaluation: https://t.co/dlnmpNvMPo

[LLM Evals Workshop @NeurIPS @LLM_eval]

See you in San Diego on December 7th!

[LLM Evals Workshop @NeurIPS @LLM_eval]

🚀 We are thrilled to announce that the LLM Eval Workshop @NeurIPSConf received 244 excellent submissions! 188 papers will be presented in poster sessions, and 5 exceptional works have been selected for oral talks.

Check out the accepted papers: https://t.co/aBFVYkCb1c

🧵👇

[LLM Evals Workshop @NeurIPS @LLM_eval]

- "The Measure of All Measures: Quantifying LLM Benchmark Quality" -- Jihan Yao, Peter Jin, Ke Bao, Qiaolin Yu et al.
https://t.co/A9Fi1sfgVB

[Huanxin Sheng @HuanxinShe5254]

I will present my #EMNLP2025 paper at the #NeurIPS2025 LLM Eval Workshop @LLM_eval (Dec. 7th 11:15 - 12:15Poster Session 2).

If you are interested in reliable LLM-as-a-judge, please come say hi! ☕️

#AI #LLM #LLMJudge #LLMEvaluation #ConformalPrediction

[Juan Miguel Navarro @JuanMiguelNC]

(5/5) Read the paper at: https://t.co/TM7qpN04NX

Looking forward to discussing more at NeurIPS in San Diego!

[Juan Miguel Navarro @JuanMiguelNC]

(1/5) My work, “LLMs Show Surface-Form Brittleness Under Paraphrase Stress Tests”, has been accepted for a contributed talk at @NeurIPSConf 2025 Evolving LLM Lifecycle: Benchmarks, Emergent Abilities, and Scaling workshop @LLM_eval
#NeurIPS #LLM #Evaluation #Robustness #AI #ML

[Riccardo Cadei @riccardocadeii]

Sketched on a few Parisian summer nights with a friend, @ChrisInterno . If you care about (causal) identification in a semi-synthetic future, we’d value your read and critique.

Preprint: https://t.co/GDN3nBlBww
Accepted at @LLM_eval workshop @NeurIPSConf

[Riccardo Cadei @riccardocadeii]

The Narcissus Hypothesis:
--Recursive training on semi-synthetic corpora enforcing human alignment induces a Social Desirability Bias: world-models (Narcissus) aim to please rather than represent, polluting data lakes and charming us (Echo) into hanging on their every word.