Top Tweets for #BenderRule
I made some slight updates to my website, inspired by the #BenderRule. This is just the first step😎
https://t.co/0UMoeVOeMt
Can LLMs be trusted for accurate factual recall?
In our recent paper, we evaluated 20 LLMs from @OpenAI, @GoogleAI, @cohere, @AIatMeta, and others using World Bank Data. We found that:
i) Even top models have factual error rates of 20-30%!
ii) Significant disparities exist across geographic and income lines in LLM performance.
Link to paper (presented at FAccT 2024): https://t.co/VlFZRWKMuV
Joint work with @EllaTabassi and @MLMazda
More details 👇
Thanks to @emilymbender for introducing the Bender rule in #NLProc that changed the colonial and anglo-centric practice. We clearly need a #BenderRule for #ICTD work.
cc: @tanmit @joyopal @ishtiaqueSIA @WorkPracticeTec @nehakumar @imadityav @kalikabali
Just witnessed the #BenderRule at #ACL2023: "what language did you use?" asks @emilymbender of a presenter to remind them that English shouldn't be assumed to be universal. Serves to reveal the 'unmarked category' in discourse.
Will reviewers now ask "where's GPT4?" We propose a #BenderRule-like rule:
That which is not open & reproducible* can't be a requisite baseline
https://t.co/qb9FfDPaLS
w @b_niranjan @LeonDerczynski @JesseDodge @alkoller @SashaMTL @MaartenSap @royschwartzNLP @nlpnoah @strubell
No contribuyas a invisibilizar otras lenguas en la tecnología. Además de que un modelo que trabaja "muy bien" en inglés, no necesariamente podría hacer lo mismo con otra lengua que no es hegemónica.
Nombrar las lenguas que se usan en los modelos es seguir #BenderRule
@so_radhikal We need a #benderrule for geography/culture. And on that note, a shameless self plug to help your case:
https://t.co/85z0IVtXmi
Check out our @aaclmeeting paper “Re-contextualizing Fairness in NLP: The Case of India”.
We analyze India-specific biases in #NLProc and propose a research agenda for meaningful fairness interventions.
https://t.co/ZMbmsDyBV7
w/ @sunipadev @partha_p_t @shachi_dave @vinodkpg 🧵
@urialon1 @aman_madaan @shuyanzhxyc @gneubig I'm curious to hear your thoughts on how well this approach may translate to non-English languages (I didn't see a mention of which languages you were working on in the paper #BenderRule) given that almost all programming languages are English-centric.
@DEGwer3456 NLPでは、英語でしか検証・実験していないのにその事実に論文で触れていないこと（英語でうまくいけばあとは同様にできるはずという暗黙の前提）、また逆に、英語以外での仕事がその言語にしか通用しないもののように見られることを問題視する立場があります。 #BenderRule
https://t.co/TWGCCEmYtH
NAACL definitely lost the 2019 #benderrule momentum. I have yet to hear someone mention that their work only covers the English language.
Do we Name the Languages we Study? The #BenderRule in LREC and ACL articles
Fanny Ducel1, Karën Fort2, Gaël Lejeune3, Yves Lepage4
1-Sorbonne Université
2-Sorbonne Université and LORIA
3-STIH
Paris-Sorbonne
4-Waseda University
Glad to see this analysis. I still regularly invoke the #BenderRule in my reviews, so it is definitely not a solved problem.
For SemEval 2021 we had it as a checkbox on the final submission page, so hopefully authors made sure to mention the language(s) in their papers!
Today at work I am attempting to build up the strength to ask a question in a presentation: specifically, which language someone trained their classifier in. My polite, semi-hedging way of indicating they broke the #BenderRule.
@NickATomlin @MadamePratolung @timnitGebru @mmitchell_ai Huh -- nice! So, the paper as a whole is #BenderRule compliant. I'd still prefer to see the language named in the abstract. The results, after all, are about Dutch (in this case) and not "natural language".
I'd guessed English on the basis of the citations to BERT, etc.
@MadamePratolung @timnitGebru @mmitchell_ai That's all I have time for now, except to note that Ctrl-F English turns up 0 hits, so #BenderRule and all that.
Check out this design, Vote Bender! https://t.co/ik5JH1RDSN
#bendergate #bender #Futurama #FANART #shirt #tshirt #benderrule
@hulu @hulu_support @TheJohnDiMaggio pay the man what he wants. #bender is only Bender with John. I love the show, but I won't watch if you recast him. Cause us #Futurama fans want the original actors! Period. #bendergate #benderrule
Controversial: what is the rational behind #benderRule?
I'm all for being explicit and acknowledge there are languages other than English.
However,
no one doubts the default language is English. What's the point?
You know, the #BenderRule is really a low bar, and yet people persist in crawling under it. Latest case in point: The LaMDA paper.
You know, the #BenderRule is really a low bar, and yet people persist in crawling under it. Latest case in point: The LaMDA paper.
#BenderRule compliance appears to be at an all-time low :/
Last Seen Hashtags on Sotwe
Most Popular Users
Elon Musk 
@elonmusk
241M followers
Barack Obama
@barackobama
119.2M followers
Cristiano Ronaldo
@cristiano
111.9M followers
Donald J. Trump
@realdonaldtrump
111.8M followers
Narendra Modi
@narendramodi
107M followers
Rihanna
@rihanna
98M followers
NASA
@nasa
92.2M followers
Justin Bieber
@justinbieber
91.2M followers
KATY PERRY
@katyperry
88.4M followers
Taylor Swift
@taylorswift13
82.3M followers
Lady Gaga
@ladygaga
73.8M followers
Virat Kohli
@imvkohli
71M followers
Kim Kardashian
@kimkardashian
70.1M followers
YouTube
@youtube
68.7M followers
Bill Gates
@billgates
64.3M followers
Neymar Jr
@neymarjr
63.9M followers
The Ellen Show
@theellenshow
62.4M followers
CNN
@cnn
61.8M followers
Selena Gomez
@selenagomez
61.5M followers
X
@x
60.8M followers