#DataContamination - Twitter Hashtag

7 months ago

Huge thanks to all authors: @MiZawalski, @mellem_boo, @balazyklaudia, @besanushi, Pablo Ribalta, and all other people involved! Read the full paper here: https://t.co/AdfMa11m1E #LLM #AI #ML #DataContamination #Benchmark #Research #NVIDIA

0

4

0

1

110

Fede_Ranaldi @FedeRanaldi

over 1 year ago

@arian_ask Impressive work! Have you considered tò test your approach on other less known benchmarks? Zero-shot Performances seems to be overestimated for famous benchmarks. See how #DataContamination impact on #TextToSQL at https://t.co/frDgmKplxu

1

2

0

90

Woojin Kim

@woojinrad

almost 2 years ago

2/ 🚨 First, when you test closed-source #LMMs like #GPT4V using data from the Web (i.e., NEJM Image Challenge), you risk #DataContamination. For example, you can directly access the Figure 1 case online. 🛑 We need to stop testing these #LLMs and LMMs on data from the Web!

woojinrad's tweet photo. 2/ 🚨 First, when you test closed-source #LMMs like #GPT4V using data from the Web (i.e., NEJM Image Challenge), you risk #DataContamination. For example, you can directly access the Figure 1 case online. 🛑 We need to stop testing these #LLMs and LMMs on data from the Web! https://t.co/FMG3x6Fx55

1

2

0

228

Fede_Ranaldi @FedeRanaldi

about 2 years ago

Our paper "Investigating the Impact of Data Contamination of Large Language Models in Text-to-SQL Translation" has been accepted at ACL Findings ! Good Job guys @HumanCentricArt #ACL2024NLP #NLproc #DataContamination https://t.co/BVrrufxB6O

0

5

2

0

306

Woojin Kim

@woojinrad

about 2 years ago

"our ﬁndings indicate that GPT-4 is contaminated with AG News, WNLI, and XSum datasets." arXiv 👉https://t.co/t8XhwecjIK #DataContamination #LLMs

Shahriar Golchin @ShahriarGolchin

over 2 years ago

Thrilled to share that our "Time Travel in LLMs" paper has been accepted to #ICLR2024 as a Spotlight! w/ my awesome advisor @msurd #LLMs #DataContamination @iclr_conf

0

10

2

1

1K

0

6

0

1

536

BusinessIntelligence

@bimedotcom

about 2 years ago

The Turing Tests of today are mistaken https://t.co/MRTwFDiaW6 By @raphaelmilliere via @IAI_TV #AI #LLMs #benchmarks #performance #TuringTest #DataContamination cc @mvollmer1 @sonu_monika @JagersbergKnut @EstelaMandela @enilev @BetaMoroney @Shi4Tech @sallyeaves @sim010101 @PerBBerggreen @ahier @gerald_bader @richardkimphd @Corix_JC @jenstirrup @dinisguarda @data_nerd @theomitsa @NeiraOsci @Hana_ElSayyed @tlloydjones @TarakRindani @Nicochan33 @FinMKTG @Fabriziobustama @Analytics_699 @LouisColumbus @YvesMulkers @HolgerGelhausen @ChuckDBrooks @SusanHayes_ @FractaloidConvo @drsharwood @jeancayeux @amalmerzouk @AndrewinContact @CurieuxExplorer @FrRonconi @FernandaKellner @mikeflache @maponi @Khulood_Almani @sminaev2015 @Eli_Krumova @pchamard @MHcommunicate @RLDI_Lamy

1

32

26

1

2K

Woojin Kim

@woojinrad

about 2 years ago

@RSNA 4/ If the authors had tried more cases, I’m sure they would have found #GPT4 failing more often than not. I took one random example from the Internet (yes, I risked #DataContamination and made it potentially easier for GPT-4), yet you can see how it fails spectacularly.

woojinrad's tweet photo. @RSNA 4/ If the authors had tried more cases, I’m sure they would have found #GPT4 failing more often than not. I took one random example from the Internet (yes, I risked #DataContamination and made it potentially easier for GPT-4), yet you can see how it fails spectacularly. https://t.co/kMCd3OjKhC

2

5

0

270

Fede_Ranaldi @FedeRanaldi

about 2 years ago

@simoneballoccu @PSchmidtova @LangoMateusz @tuetschek Very considerable work! DC is an intriguing problem not only for evaluation but also witnesses the ambiguity between learning and memorization. You'll probably find interesting our work that is deeply dedicated to Text-to-SQL. #datacontamination #text2sql https://t.co/BVrrufxB6O

0

3

0

83

Fede_Ranaldi @FedeRanaldi

about 2 years ago

@Adhiguna_AIaaS Good thread. Another advantage of this solution is the easiness of interaction with the model (just prompt!) . But can be Benchmarks on very known Test Sets affected by Data Contamination? #datacontamination #overestimation Take a look at our work https://t.co/BVrrufxB6O

0

2

0

41

Shahriar Golchin @ShahriarGolchin

over 2 years ago

Thrilled to share that our "Time Travel in LLMs" paper has been accepted to #ICLR2024 as a Spotlight! w/ my awesome advisor @msurd #LLMs #DataContamination @iclr_conf

Shahriar Golchin @ShahriarGolchin

almost 3 years ago

Data contamination suggests LLMs have possibly seen test data from downstream tasks. Our recent study introduces a novel method to replicate LLMs' training data, including downstream dataset instances, to aid in detecting data contamination. Read more: https://t.co/3Ll3NcSJDr

1

16

5

6

2K

0

10

2

1

1K