Maximilian Böther

making data loaders go brr | mts @datologyai | Ph.D. student @ETH_EN @SystemsGroupETH @anaklimovic | previous gigs at @HPI_DE @apple @google

[Lucas Beyer (bl16) @giffmana]

Of course, love me some confirmation bias!

DCLM was for text, DCVLM is the same for vision: analyzing VLM data mix across scales. Filtering is no bueno, but the "type" mixing matters, with unfortunately (but imo expectedly) the best small-scale mix != the best mid-scale mix.

[Maximilian Böther @MaxiBoether]

@HKydlicek Well people don‘t love to run Ray on Slurm but especially national HPC clusters won‘t move away from Slurm any time soon😅

[Maximilian Böther @MaxiBoether]

DataComp-VLM aka DCVLM is finally out! It was such a pleasure working with this team of cracked ML data folks. Check out what changes in comparison to DCLM as we have surprising findings for our VLM data pool.

Models are what they eat, and it continues to show!

[Vishaal Udandarao @vishaal_urao]

🚀New Paper
https://t.co/cHemp66IDM

Everyone obsesses over VLM architectures & training recipes. But what about the data?

Presenting the latest work in the DataComp-series: a testbed for VLM data curation with 1,000+ controlled experiments and some surprising lessons 👀

🧵👇

[Thorsten Ball @thorstenball]

There are very few things that have radicalized me as much as founding a UG in Germany

[Jenia Jitsev 🏳️‍🌈 🇺🇦 🇮🇱 🇮🇷 @JJitsev]

The misery around recent decision for EU Frontier AI Grand Challenge reveals again strong deficits in EU decision making and its lack of basic competence in ML/AI. Hard to explain otherwise how an entity without a track of record in NeurIPS/ICLR/ICML gets to do "Frontier AI".

[Charles 🎉 Frye @charles_irl]

wild to me that people vibe-generate slides for conference talks

they are ugly (for now). they are low info densiry (thanks rlhf)

but worse, they don't represent your thoughts, so your presentation of them will be terrible, unless you put in a ton of work (so just write them!)

[Maximilian Böther @MaxiBoether]

@HKydlicek At @datologyai we are currently building a new dataloader - currently in private beta. Just DMd you!

[Markus Wulfmeier @m_wulfmeier]

A key reason for Zürich's recent robotics and AI boom. There's just incredible talent hidden in ETH, EPFL, UZH and many other places!

[Cody Blakeney @code_star]

Poor one out for the homies

[Christina Baek @_christinabaek]

Models are typically specialized to new domains by finetuning on small, high-quality datasets.

We find that repeating the same dataset 10–50× starting from pretraining leads to substantially better downstream performance, in some cases outperforming larger models. 🧵

[Ricardo Monti @RicardoMonti9]

1/ People often think better multilingual models must come at the cost of English performance. Not true. The constraint isn’t capacity, it’s data quality, and we can fix it.

Today @datologyAI shares ÜberWeb: a year of multilingual curation lessons, scaled to 20T+ tokens.

[Maximilian Böther @MaxiBoether]

@xeophon @RicardoMonti9 @datologyai > You even got one in your name, how fun

Until you start applying for a US visa/bank account/SSN, this is where the fun stops 😂

> ß/ẞ into the data as well

As part of my PhD contract at ETH Zurich I had to agree to not use those anymore :( Swiss rules and whatnot 🇨🇭

[Maximilian Böther @MaxiBoether]

@RicardoMonti9 @xeophon @datologyai Intern-driven data curation (TM)

He keeps telling me we „look at the data“, „thats your job“

[Maximilian Böther @MaxiBoether]

@RicardoMonti9 @xeophon @datologyai ricardo made me manually review each and every of them😩

[Maximilian Böther @MaxiBoether]

@RicardoMonti9 @chipro ack

[Squirtle @squirtle_says]

the world you remember no longer exists

[Maximilian Böther @MaxiBoether]

@StasBekman @datologyai @josh_wills I sent over a mail to your Snowflake address, probably easier than a tweet. In any case, glad the OVERLORD ref was already helpful!

[Maximilian Böther @MaxiBoether]

@leavittron Would this be the right time and place to re-negotiate my intern stipend?

[Maximilian Böther @MaxiBoether]

@leavittron I forgive you