New blog post: BERT Busters👻: outlier dimensions that disrupt Transformers
https://t.co/HQLkuFKrVy
TLDR: We found the Achilles' heel of BERT&co! Disabling just <0.0001% of outlier layer output weights drops model quality this much:
/1
Hi #ACL2021nlp, check out our new work “EmailSum: Abstractive Email Thread Summarization” (11-12p ET Aug4 Wed 15D). 😀
w @real_Asli@JianfengGao0217@MohitBan47 (@uncnlp+@MSFTResearch)
https://t.co/estXVP6b7N
Github: https://t.co/tBQJ2Nyru7
Video: https://t.co/enSOSBYpXV
🧵⬇️
Is your summarization model hallucinating 👁️? Is it truly abstracting🧠? Or just extracting✂️?
Next week at ACL, stop by our demo of SummVis, an interactive vis tool to help you answer these questions.
Paper/code/demo: https://t.co/rGcb2KiIQu
@SFResearch@StanfordAILab
1/N
Reconstructing Implicit Knowledge with Language Models.
Generating statements that explicate implicit knowledge connecting sentences in text.
https://t.co/QKz09sV62Z
https://t.co/wyGiWVcVkC
Happy to share that our paper "Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation" got into #NAACL2021!🎉Joint work w/ @StDario1 and Alex Fraser.
Here is a summary thread👇
A great article (except for the screen-filling advertisements) on how AI is used successfully by Google and NASA for discovering 2 new planets. https://t.co/nO5z7m2PN8