Jesse Vig

AI Researcher

[Jesse Vig @jesse_vig]

Congratulations @meetdavidwan for this great collaboration between Salesforce Research and UNC! @JotyShafiq @mohitban47

[David Wan @meetdavidwan]

How does positional bias influence faithfulness in long-form summarization (LFSumm)? 🤔

In our latest study, we analyze the effect of positional bias on faithfulness metrics and generated summaries, and methods to mitigate this bias.

📄Paper: https://t.co/3WV1kcyiAq

Thead🧵👇

[Jesse Vig @jesse_vig]

@YesThisIsLion @tunguz Can't quite place it...

[Philippe Laban @PhilippeLaban]

Excited to share this fun new work on the 🩴FlipFlop Effect.

In short: if you ask models if they're sure of their answers, they tend to change their minds (and severely degrade accuracy).
What's mindblowing is how universal the effect is across LLMs (GPTs, Gemini, Claudes, …).

[Caiming Xiong @CaimingXiong]

Excited to share a new preprint on the 🩴FlipFlop Effect.
We prompt LLMs with a classification task, and challenge the model by following up with “Are you sure?”. The model can confirm or flip its answer. The results? More flips than a gymnastics competition! 🤸‍♂️ 1/N

[Jesse Vig @jesse_vig]

@krandiash @HazyResearch Congrats!

[Jesse Vig @jesse_vig]

@YesThisIsLion Thank you Llion!

[Jesse Vig @jesse_vig]

How can we teach models to simplify text using the revision history of Wikipedia articles?

Check out our paper "SWiPE: A Dataset for Document-Level Simplification of Wikipedia Pages" presented by @PhilippeLaban at #acl2023NLP (poster session 5). 🎉

[Caiming Xiong @CaimingXiong]

🤔Which words in your prompt are most helpful to language models? In our #ACL2023NLP paper, we explore which parts of task instructions are most important for model performance.
🔗 https://t.co/kzhbzcEnHA
Code: https://t.co/tCBtzH3A5M

[Jesse Vig @jesse_vig]

Check out our latest work on text simplification at #acl2023nlp .

[Philippe Laban @PhilippeLaban]

Very excited to present SWiPE in person at ACL in a few weeks.

In short, we collaborated with Wikipedia editors to understand the process of document simplification and take a (small) step towards improving document accessibility by releasing a large dataset!

[WikiResearch @WikiResearch]

"SWIPE: A Dataset for Document-Level Simplification of Wikipedia Pages" leveraging the entire revision history when pairing enwiki/simplewiki pages, to identify simplification edits.

(Laban et al, 2023)

https://t.co/67FEP0uUWY
@iam_wkr

[Wojciech Kryściński @iam_wkr]

Check out our work on text simplification! SWiPE accepted at #ACL2023

[Caiming Xiong @CaimingXiong]

By aligning Wikipedia articles to their simplified versions on Simple Wikipedia, we reconstruct the process by which human editors simplify whole documents, in contrast to prior work focused on sentence-level simplification.

[Caiming Xiong @CaimingXiong]

Finding a document too dense to decipher? 🤔Content a bit convoluted? Essay too esoteric? Check how we simplify and improve document readability using SWiPE. Join us in making knowledge accessible to all! 🌐
🔗Paper: https://t.co/1ebelhoWW9
🔗Github: https://t.co/uMVZnXmQuX

[Yixin Liu @YixinLiu17]

Delighted to announce our paper has been accepted for an oral presentation at #ACL2023 oral! In this work we emphasize the intricate complexity of human evaluation while it is becoming even more crucial for both model training and evaluation in the LLM era.

[Jesse Vig @jesse_vig]

How can NLP help us understand the diversity of news coverage of a topic? Check out the latest work from @PhilippeLaban et al. appearing at #CHI2023 this week.

[Philippe Laban @PhilippeLaban]

📰 How can we make it easier for news readers to access nuanced and diverse coverage from multiple sources?

In our #CHI2023 paper, we propose to highlight news coverage diversity through generated *discord questions* shown to the readers.

Link: https://t.co/rcWIjuvWag

[Wojciech Kryściński @iam_wkr]

Very excited to have the opportunity to present research done at @SFResearch on Automatic Text Summarization at @ZIL_IPIPAN

„Long Story Short: A Talk about Text Summarization” will cover the current state of the field, existing challenges, and future directions.

[Chien-Sheng (Jason) Wu @jasonwu0731]

Human preference != Job Done. Check our interesting findings (ROSE🌹) on summarization!

Thanks to my fantastic collaborators @alexfabbri4 @YixinLiu17 @stefan_fee Yilun Zhao Linyong Nan Ruilin Han @HanSineng @JotyShafiq @CaimingXiong @dragomir_radev!

[Alex Fabbri @alexfabbri4]

You can explore the ACU annotations in Rose🌹along with protocol results on our demo page and start using our dataset!

Repo: https://t.co/crPTeZepL4
Demo page: https://t.co/ZIPPSInw5a
Dataset: https://t.co/g1CSyxw1qF

[Alex Fabbri @alexfabbri4]

🚨🆕📄🚨
How gold is your human evaluation? We seek the answer, and its implications in the GPT3 era, in our preprint “Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation”

Paper: https://t.co/Y2LCITvpyM
Equal contribution @YixinLiu17