Matthias Gerstgrasser

I teach AIs to play nice with each other, and sometimes with us humans.

[Rylan Schaeffer @RylanSchaeffer]

📢New preprint📢

🔄Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World 🔄

A deeper dive into the effects of self-generated synthetic data on model-data feedback loops

w/ @JoshuaK92829 @ApratimDey2 @MGerstgrasser @rm_rafailov @sanmikoyejo

1/9

[Zack Witten @zswitten]

On one end of the line: ELIZA, the psychotherapist from the 60s. First chatbot to make people believe it was human. Rulebound, scripted, deterministic. Still around on the web.

On the other end of the line: yr favorite LLM.

How will they react? Will they know?

[Matthias Gerstgrasser @MGerstgrasser]

@shi_weiyan @aclmeeting Congratulations, that's amazing! And so well-deserved.

[Rylan Schaeffer @RylanSchaeffer]

I sadly couldn't attend #ICML2024 but our paper received ✨Outstanding Paper ✨ at the @TiFA_ICML2024 workshop!

📈🤔 Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive? 📈🤔

Why should you care?

1/N

[Workshop on Data-centric Machine Learning Research @DMLRWorkshop]

thank you authors, reviewers and speakers for your contributions to a great DMLR @icmlconf

presentations by @AdtRaghunathan @BlancheMinerva @giffmana @AlexGDimakis @nomic_ai @MGerstgrasser @angelinepouget at
https://t.co/ZIzbmcP5bp

join & contribute at https://t.co/4lLIcWfhUu

[Matthias Gerstgrasser @MGerstgrasser]

@NotGive39111604 @RylanSchaeffer Yes, that is different! Adding 9 tokens of synthetic data per token of real data is fine (that follows from our work). Deleting 90% of real data and replacing it is bad (follows from previous papers). What matters is how much real data you keep, not how much synth data you add.

[Matthias Gerstgrasser @MGerstgrasser]

@rouli @RylanSchaeffer @YangjunR @moinnadeem @sj_manning @gabemukobi Nothing! This happens in our model in iteration 9 and onwards, as we add as much synthetic data in each iteration as we have real data to begin with. You can even show that adding this much synthetic data in the first iteration doesn't break things.

[Matthias Gerstgrasser @MGerstgrasser]

@hausdorff_space @RylanSchaeffer @YangjunR @moinnadeem @sj_manning @gabemukobi My understanding is that the xrisk argument is very different, and more about models coding their own architecture improvements, not about training on recursively generated data.

[Matthias Gerstgrasser @MGerstgrasser]

@YangjunR @RylanSchaeffer @moinnadeem @sj_manning @gabemukobi To your second question, we actually looked at this! With just increasing dataset size alone (but fully synthetic data from the latest iteration) you still see some collapse with language models.

[Matthias Gerstgrasser @MGerstgrasser]

@YangjunR @RylanSchaeffer @moinnadeem @sj_manning @gabemukobi My understanding is that it's entirely the amount of real data that you keep. If you only keep 30% of it, of course you see degradation. But if you keep all of it, then you don't need to also increase the dataset size with synthetic data. (We just show it's OK if that happens.)

[Matthias Gerstgrasser @MGerstgrasser]

@gabemukobi @RylanSchaeffer @sj_manning @moinnadeem @YangjunR @alexandr_wang To be clear, we keep *all* of the original data. One take-away from our paper is that it's the total amount of real data that matters, not what fraction of the dataset it constitutes. Thinking about percentages can obscur that.

[Matthias Gerstgrasser @MGerstgrasser]

Joint work with @RylanSchaeffer
@ApratimDey2
@rm_rafailov
@sanmikoyejo
@danintheory
@Andr3yGR
@Diyi_Yang
David Donoho @rajashree
@tomekkorbak
@sleight_henry
@McHughes288
@dhruv31415

3/3

[Matthias Gerstgrasser @MGerstgrasser]

Are you worried that training your generative AI models on synthetic data will lead to bad things happening? Are you at ICML this week? Join us at the DMLR and FM-WILD workshops, where I'll be talking about our most recent work on model collapse, and how there is hope yet! 1/n

[Matthias Gerstgrasser @MGerstgrasser]

Friday 14:30: FM-WILD poster session
Saturday 11:00: DMLR panel discussion
Saturday 14:45: DMLR talk

Preprint: https://t.co/D79tT6EmSZ

2/n

[Matthias Gerstgrasser @MGerstgrasser]

Huge thanks to my fantastic co-authors @RylanSchaeffer @ApratimDey2 @rm_rafailov @sanmikoyejo @danintheory @Andr3yGR @Diyi_Yang David Donoho @rajashree @tomekkorbak @sleight_henry @McHughes288 @dhruv31415

[Matthias Gerstgrasser @MGerstgrasser]

Excited that our paper on model collapse has been accepted @COLM_conf

Is model collapse inevitable? As we show: No! It may well not be as catastrophic as feared in practice.

Preprint: https://t.co/D79tT6EUIx

[Rylan Schaeffer @RylanSchaeffer]

Woke up to find out our paper was accepted at @COLM_conf !!

📉📉 Is Model Collapse Inevitable? 📉📉

No! Model collapse is largely avoided if data accumulate over model-fitting iterations

Previous papers found model collapse by discarding all data after each iteration

1/2

[Matthias Gerstgrasser @MGerstgrasser]

Really amazing work by awesome people. Very excited to see what people will make with this.

[Jeff Hawke @jeffrey_hawke]

Super excited to share what I’ve been working on: @odysseyml!

We’re building Hollywood-grade visual AI, to enable storytellers to create new, amazing movies, TV shows, and video games.

We believe this technology is what comes after text-to-video. More below.👇

[Matthias Gerstgrasser @MGerstgrasser]

UoE's fantastic multi-agent RL talk series is now available on youtoube for free - fantastic resource for anyone wanting to get up to speed with the latest research. (Including my own talk on SUPER: https://t.co/H1NhUVHVvf )

[Stefano V. Albrecht @s_albrecht]

(1/2) Exciting news - our RL Reading Group meetings will now be on YouTube 🎉 Subscribe to stay up-to-date with SOTA RL research: https://t.co/aarAkJE43E

Our (virtual) meetings are open to all, sign up here! https://t.co/CzLRfPpAlO

[Matthias Gerstgrasser @MGerstgrasser]

Excited to be speaking at the AI Talk Series about Generative AI and LLMs in San Mateo tonight!

[Mariane Bekker @marianebekker]

Join our Gen AI and LLM talk today in San Mateo to connect with fellow AI founders and engineers from Stanford, Harvard and MIT building the future:

https://t.co/sYprOOz6Vu

[Rylan Schaeffer @RylanSchaeffer]

@nvidia announces Nemotron 340B, designed explicitly for producing synthetic data to train other models!

https://t.co/fuc0rOqI8j

But won't this cause future models to go MAD?? The curse of recursion, right?

Probably not 👇👇👇