David Esiobu

[Dieuwke Hupkes @_dieuwke_]

New deep-dive into evaluation data contamination 😍🤩.

Curious how much contamination there really is in common LLM training corpora, how much that actually impacts benchmark scores and what is the best metric to evaluate that? Read our new preprint!

https://t.co/xcu3NNLZb0

[AI at Meta @AIatMeta]

Starting today, open source is leading the way. Introducing Llama 3.1: Our most capable models yet.

Today we’re releasing a collection of new Llama 3.1 models including our long awaited 405B. These models deliver improved reasoning capabilities, a larger 128K token context window and improved support for 8 languages among other improvements. Llama 3.1 405B rivals leading closed source models on state-of-the-art capabilities across a range of tasks in general knowledge, steerability, math, tool use and multilingual translation.

The models are available to download now directly from Meta or @huggingface. With today’s release the ecosystem is also ready to go with 25+ partners rolling out our latest models — including @awscloud, @nvidia, @databricks, @groqinc, @dell, @azure and @googlecloud ready on day one.

More details in the full announcement ➡️ https://t.co/hhJoLm5eLV
Download Llama 3.1 models ➡️ https://t.co/rRjvmxqCTC

With these releases we’re setting the stage for unprecedented new opportunities and we can’t wait to see the innovation our newest models will unlock across all levels of the AI community.

[David Esiobu @davides_io]

@DanielleFong Why isn’t there more grid scale storage?

[AI at Meta @AIatMeta]

Introducing Meta Llama 3: the most capable openly available LLM to date.

Today we’re releasing 8B & 70B models that deliver on new capabilities such as improved reasoning and set a new state-of-the-art for models of their sizes.

Today's release includes the first two Llama 3 models — in the coming months we expect to introduce new capabilities, longer context windows, additional model sizes and enhanced performance + the Llama 3 research paper for the community to learn from our work.

More details ➡️ https://t.co/nFll4exicO
Download Llama 3 ➡️ https://t.co/Ps0OAHt0RR

[David Esiobu @davides_io]

@nope_its_lily Find an interesting paper and pick things from it to replicate. E.g. can you reproduce the eval results from the BloombergGPT paper? https://t.co/g2JDpkgf92 HF has good APIs and docs for doing most of the steps required.

[David Esiobu @davides_io]

@firepile @OpenAI I’d guess there’s some kind of output filter preventing it from repeating training data. The authors here found similar behavior in some of their tests: https://t.co/Nr9U2quBKe

[David Esiobu @davides_io]

@DataChaz @mmitchell_ai @nazneenrajani @natolambert @SanhEstPasMoi @Thom_Wolf Small typo, the OPT paper reported 180B tokens in the pertaining corpus and not 100B (section 2.3 of https://t.co/ONxSWg5Cka)

[Brendan Dolan-Gavitt @moyix]

ChatGPT exploits a buffer overflow 😳

[David Esiobu @davides_io]

@DemocratBased @yishan @sriramk @elonmusk This. There are loads of conf talks by Twitter engineers about their architecture on YouTube. Some a bit dated but the bones are there

[David Esiobu @davides_io]

@chrisdotio @RobGuinness @deliprao also the JS and TS implementations for it look really different. https://t.co/oJoNCpiy9k

[David Esiobu @davides_io]

@chrisdotio @RobGuinness @deliprao a lot of the difference seems to have come from one benchmark (fannkuch-redux): https://t.co/L5JMpifZLd

there's also incomplete language coverage for some benchmarks...not clear how that's handled in calculating the final results

[David Esiobu @davides_io]

@WillManidis https://t.co/cGAa8uzY8C

[David Esiobu @davides_io]

@davidfowl Are batching and sharding on there?

[David Esiobu @davides_io]

@kelseyhightower Good CLIs are way underrated. A good interface + jq, and I can stitch together the rest to build my own workflows

[David Esiobu @davides_io]

@LynneDubois @CupofJOE_ @designmom This thread has some of the recipes people used over the years but the takeaway I got was that they’re generally not nutritionally complete

[David Esiobu @davides_io]

@LynneDubois @CupofJOE_ @designmom https://t.co/5r1xaNUITy

[Carla Cevasco, PhD @Cevasco_Carla]

You may be hearing the argument that before the rise of modern commercial infant formula, babies all ate breastmilk and everything was great. As a historian of infant feeding, let me tell you why that’s not true.

[David Esiobu @davides_io]

@Manueldtc @Kujokaii @itsstilltru What you need to do is just keep moving. Never stand still long enough for anyone to notice and you’ll be fine

[David Esiobu @davides_io]

@nickchapsas Not specifically F# advice but reading open source projects in a problem space you understand already is another way to tackle a new language

[David Esiobu @davides_io]

@borud @rakyll +1. Clean navigation for all references to/from a call site, recursively, in either direction would get you half the way there I think. Some IDEs (e.g. IntelliJ) do this with a collapsible tree view but would be nice to have while reading in the browser

[David Esiobu @davides_io]

@colmmacc brb throwing my phone in a river