Dynabench

Official dynabench account

[Douwe Kiela @douwekiela]

Progress in AI continues to outpace benchmarks.

Check out this new plot, inspired by @DynabenchAI, that shows just how quickly it's happening.

Read more about it here: https://t.co/MwmfOdUy0B

[Dynabench @DynabenchAI]

Big congrats to longterm Dynabenchers @douwekiela and @apsdehal on their exciting launch! Looking forward to seeing what you build! 🎉

[Douwe Kiela @douwekiela]

Super excited to announce that @apsdehal and I have launched a new company: @ContextualAI!

Why did we start it? Because LLMs are going to radically change the way enterprises operate, and we see a huge need for LLMs that actually work for enterprise use cases.

1/5

[Dynabench @DynabenchAI]

@DADCworkshop @MLCommons 📣Big shout out to @23rd_Conspiracy @ciropython 📣 for bringing all these exciting updates to the platform! We’re excited to share the future of Dynabench with you!
7/7

[Dynabench @DynabenchAI]

At Dynabench, we're gearing up for the AI race, and we embrace the rapid pace of change! We're excited to announce some big updates that make our new-and-improved platform faster, better, and easier to use, for leaderboard users, dataset creators, and benchmark owners! 1/7

[Dynabench @DynabenchAI]

@DADCworkshop @MLCommons We've also streamlined the process for leaderboard users! Our new, easy-to-use templates that can support many kinds of challenges, and any python-based machine learning framework (tensorflow, pytorch, sklearn etc). Log in to learn more about it here: https://t.co/BhrUsERo6q
6/7

[Dynabench @DynabenchAI]

The first round of Dataperf 2023 challenges are open for participation through May 26th, 2023! Challenge winners will have the opportunity to present their work @icmlconf #ICML2023! Visit https://t.co/92tzOxG9Br to learn more and get involved!

[Dynabench @DynabenchAI]

We are excited to welcome the #DataPerf community to the Dynabench platform! DataPerf consists of 5 data-centric challenges spanning data selection, cleaning, generation and valuation across speech, text and vision modalities. Head over to https://t.co/kVpDyttKg5 to explore them!

[AI at Meta @AIatMeta]

Today, Meta researchers together with @MLCommons working group, are launching DataPerf, the first platform for building data & data-centric AI algorithm leaderboards.

We're excited for how DataPerf will help to push the data-centric AI field forward ⬇️

[Aaron Mueller @amuuueller]

Announcing the BabyLM 👶 Challenge,
the shared task at @conll_conf and CMCL'23!

We’re calling on researchers to pre-train language models on (relatively) small datasets inspired by the input given to children learning language.

https://t.co/w6LxYAgnyA
https://t.co/EGgmK8ZORq

[Dynabench @DynabenchAI]

@rebeccatqian The Dynabench project welcome new ideas for metrics! If you have a cool idea for better measuring fairness, robustness, throughput, memory, performance, or anything else, we're listening! Feel free to reach out, or submit a PR to have your metric included on the platform! 2/2

[Dynabench @DynabenchAI]

Exciting new work on fairness at #NeurIPS2022: @rebeccatqian will present Perturbation Augmentation for Fairer NLP tomorrow at the RobustSeq WS (https://t.co/jpmBoRXMQu). We're excited to apply this work to improve fairness measurement on Dynabench! You won't want to miss it! 1/2

[Mubashara Akhtar @akhtarmubashara]

Great talk today by @max_nlp from @ucl_nlp @CohereAI on Dynamic Adversarial Data Collection at the @kclinformatics Distributed AI group!

[GenBench @GenBench]

Ever wanted to know more about generalisation in NLP but overwhelmed with the number of papers on ArXiv? Fear not! We read 400+ papers, 600+ experiments, and designed a taxonomy 📝 to categorise the research for you! (1/n) 🧵

https://t.co/qvftIztAWq

[Adina Williams @adinamwilliams]

How can we improve benchmarking? The @DynabenchAI experiment aims to make faster progress with dynamic data collection, and today, we are pleased to introduce our next stage: @MetaAI has funded 5 exciting research proposals on the theme of "Rethinking Benchmarking”! Congrats to:

[Dynabench @DynabenchAI]

We are very excited to be teaming up with @MLCommons! Stay tuned for our upcoming new datasets and other updates!

[MLCommons @MLCommons]

Today @MLCommons has formed a Working Group to develop the open-source Dynabench Platform to support benchmarking of datasets, data-centric algorithms & models. Join us in helping to build the data-centric AI community! Visit https://t.co/kKlMwE66GJ

[Davis Blalock @davisblalock]

"DataPerf: Benchmarks for Data-Centric AI Development"

What if instead of holding the data constant and benchmarking different models, we held the model constant and benchmarked different data pipelines? [1/7]