GenBench

State-of-the-art generalisation testing in NLP. Tag us for a RT of your NLP generalisation paper tweet!

[GenBench @GenBench]

The GenBench workshop is back! Do you work on generalisation (benchmarking) in #NLProc? Submit to the 2nd edition (https://t.co/XqMMYRW8vQ) co-located with #EMNLP2024. We have a regular track and a ✨collaborative benchmarking task (CBT)✨ that's fully LLM-focused this year (1/6)

[GenBench @GenBench]

@robinomial @mrdrozdov @_dieuwke_ @najoungkim @kylelostat @sameer_ Two independently arrived at but similar conclusions 😁

[GenBench @GenBench]

That's a wrap! We (@glnmario, @christos_c, @_dieuwke_, @vernadankers, @khuyagbaatar_b, @a_kazemnejad & @ryandcotterell) thank all presenters, authors, reviewers and attendees!! The keynotes, the cats 😻, the posters, the talks and the lively panel: it was fantastic👏 🔥

[GenBench @GenBench]

@mrdrozdov @_dieuwke_ @najoungkim @kylelostat @sameer_ @robinomial We're discussing, our initial response: reward hacking is a subset of overfitting, but also, what do you mean with reward hacking? 😁

[GenBench @GenBench]

@mrdrozdov @_dieuwke_ @najoungkim @kylelostat @sameer_ @kylelostat @sameer_ @robinomial any thoughts? 😁

[Najoung Kim 🫠 @najoungkim]

so proud of @HayleyRossLing for getting a best paper award at @GenBench this year!! 🎉🪅🎉 I'm sure @TeaAnd_OrCoffee would be too :) check out our paper and share if you think homemade cats are cats!

[Kanishka Misra 🌊 @kanishkamisra]

Woohoo go tinlab! Congrats @HayleyRossLing @TeaAnd_OrCoffee @najoungkim!!

[GenBench @GenBench]

Congratulations!

[Najoung Kim 🫠 @najoungkim]

so proud of @HayleyRossLing for getting a best paper award at @GenBench this year!! 🎉🪅🎉 I'm sure @TeaAnd_OrCoffee would be too :) check out our paper and share if you think homemade cats are cats!

[GenBench @GenBench]

Congrats to all the authors!

[GenBench @GenBench]

Closing remarks and best paper award by @vernadankers

[GenBench @GenBench]

Best paper!

[GenBench @GenBench]

And we also have an honourable mention!

[GenBench @GenBench]

Come listen to the hot takes of our panelist in the Brickell room! Do we still need generalisation evaluation? 🧐 #GenBench2024 #EMNLP2024

[GenBench @GenBench]

Still at the poster session? Come join us for keynote 3 by @sameer_!

[GenBench @GenBench]

Did you miss the GenBench poster session? Don't worry we've got you, here are (nearly all) posters! 😉 #GenBench2024 #EMNLP2024 Next up: keynote by Sameer Singh at 3!

[GenBench @GenBench]

Last spotlight presentation:

MMLU-SR: A Benchmark for Stress-Testing Reasoning Capability of Large Language Models

https://t.co/4pyv01TbWE

Unfortunately the authors couldn't make it, the work is kindly presented by their colleague Hengyi Wang 🙏

[GenBench @GenBench]

Spotlight time! Mirella Bueno on

MLissard: Multilingual Long and Simple Sequential Reasoning Benchmarks

https://t.co/ARmGeONz2c

[GenBench @GenBench]

Continuing with Bastian Bunzeck, presenting

The SlayQA benchmark of social reasoning: testing gender-inclusive generalization with neopronouns

https://t.co/70kDItm3BB

[GenBench @GenBench]

@kylelostat Plus more cat pictures! 😻😻

[GenBench @GenBench]

Join us for our second keynote by Olmo co-lead @kylelostat

[GenBench @GenBench]

@kylelostat He got all the room snickering already at slide 3! 😁