Performance on CoDA21 improves linearly with model size. However, even the largest 170B parameter model performs still poorly (<50% acc.), while expert humans can solve the task perfectly. Solving CoDA21 takes time and is not really well suited for crowd-sourced evaluation.
The paper for BIG-bench🪑is finally live at https://t.co/ShujvO3yzK. It contains 204 diverse and challenging language tasks, including our context definition alignment task, https://t.co/r52ZRcwOSg.
A nice summary thread for BIG-bench by @jaschasd:
https://t.co/iQgiyI0jMP
After 2 years of work by 442 contributors across 132 institutions, I am thrilled to announce that the https://t.co/wezEGzDEHt paper is now live: https://t.co/4Yg36EB9Ru. BIG-bench consists of 204 diverse tasks to measure and extrapolate the capabilities of large language models.
After 2 years of work by 442 contributors across 132 institutions, I am thrilled to announce that the https://t.co/wezEGzDEHt paper is now live: https://t.co/4Yg36EB9Ru. BIG-bench consists of 204 diverse tasks to measure and extrapolate the capabilities of large language models.
Excited to share that our paper “Towards a Broad Coverage Named Entity Resource: A Data-Efficient Approach for Many Diverse Languages" has been accepted at #LREC2022! 🎉
Joint work with @imani_ayyoob, @PDufter, and @HinrichSchuetze. [1/3]
Preprint: https://t.co/sX9GR3G0eU
Our work (w/ @HinrichSchuetze and @4digitaldignity) collecting #HateSpeech data in collaboration with affected communities around the globe has been accepted to #ACL2022 Findings. [1/3]
⚙️ https://t.co/nFCzUv5P9c
📝 https://t.co/O12ddiouj7
1- In collaboration with Schütze Lab (LMU), Icon Lab (Bilkent) and ASELSAN Research Center, KocLab has published "Learning Interpretable Word Embeddings via Bidirectional Alignment of Dimensions with Semantic Concepts" in Information Processing and Management journal. The paper-
7/ CoDA21 is publicly available at https://t.co/h9vIbG7ObO
Do you have a cool model with strong NLU? Go ahead and test it on CoDA21 to see if it can compete with humans.
Looking forward to meeting you at #ACL2022, hopefully in person.
I am thrilled to share that our paper "CoDA21: Evaluating Language Understanding Capabilities of NLP Models With Context-Definition Alignment
" got accepted to #ACL2022. Joint work with @timo_schick and Hinrich Schütze.
https://t.co/v1enLGPk4e
Below is a summary thread👇
6/ Performing well on CoDA21
requires detailed understanding of contexts, performing complex inference, and having world
knowledge, which are crucial skills for NLP. Our results indicate a lack of these skills in the current state of the art in NLP.
Check our new paper at #EMNLP2021 "Graph Algorithms for Multiparallel Word Alignment". We get better results compared to bilingual aligners even when we use noisy translations.
Joint work: @imani_ayyoob@lksenel@PDufter@HinrichSchuetze
PDF: https://t.co/rCoOvgV8dj
#NLProc
🎉 Happy to share that "It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners" (w/@HinrichSchuetze) has been accepted to #NAACL2021#NLProc 🥳
📷 Camera-ready: https://t.co/73RPkN2FuD
🖥️ Code: https://t.co/YCrBnggof7
📝 Blog: https://t.co/YdCgsopk7w
Happy to share that our paper "Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation" got into #NAACL2021!🎉Joint work w/ @StDario1 and Alex Fraser.
Here is a summary thread👇
🎉New paper🎉 In "Self-Diagnosis and Self-Debiasing", we investigate whether pretrained LMs can use their internal knowledge to discard undesired behaviors and reduce biases in their own outputs (w/@4digitaldignity + @HinrichSchuetze) #NLProc
Link: https://t.co/PQ04Ly3eFd [1/3]
I am happy to share that our work on evaluating word understanding of pretrained language models using dictionary definitions was accepted at EACL 2021.
Joint work with @HinrichSchuetze
Preprint: https://t.co/VMSvWA61wg