EvalAI

Evaluating state-of-the-art in AI

[EvalAI @eval_ai]

Submissions now open on EvalAI for #AlexaPrize SimBot Challenge from @AmazonScience 🎉
Prizes worth $500K, $100K, and $50K for winners!

[Amazon Science @AmazonScience]

Today, the #AlexaPrize SimBot Public Benchmark Challenge starts! It's the first time Amazon has welcomed teams other than those sponsored by the company to take part in the challenge. Visit the @eval_ai website for more information, and how to participate: https://t.co/QpZ8wemHqG

[Rohit Girdhar @_rohitgirdhar_]

Excited to share a look-back on the Ego4D challenges 2022! In spite of being a brand new dataset, we had strong uptake and healthy competition, leading to significant gains over our baselines (upto 300%!). Check out our analysis and join us in 2023 :) https://t.co/GSY7Vbf292

[EvalAI @eval_ai]

New milestone: EvalAI now hosts 100+ active challenges! From 1 challenge (VQA) in 2017 to here in 5 years:

- 200+ challenges
- 18k+ users
- 180k+ submissions
- 30+ organizations

[Rishabh Jain @rishabhjain2018]

New year, new user @AmazonScience and new challenge on @eval_ai 🎉. Can't be a better start for 2022!

[Awesome Machine Learning Repositories @MLRepositories]

EvalAI: :cloud: :rocket: :bar_chart: :chart_with_upwards_trend: Evaluating state of the art in AI
Lang: Python
⭐️ 1290
#MachineLearning
Author: @eval_ai
https://t.co/FAHI14jxhA

[Chethan Pandarinath @chethan]

We provide datasets in the Neurodata Without Borders @NeurodataWB format, accessible via @DANDIarchive. We host benchmarks and leaderboards on the @eval_ai platform, with private held-out data for standardized model evaluation.

[EvalAI @eval_ai]

Excited to host BEHAVIOR Challenge 2021. Visit https://t.co/04T4BAPHHu to participate. 📈

[Fei-Fei Li @drfeifei]

Drum roll! We release BEHAVIOR, an embodied AI benchmark and challenge of 100 complex household activities that would pose significant difficulties to all of today’s state-of-the-art algorithms. Hope to see great participation to this challenge @ICCV_2021

[roboticvisionchallenge @robVisChallenge]

We are happy to announce that our #sceneunderstanding challenge for #CVPR21 has been extended until May 31st.
Submissions accepted on @eval_ai
https://t.co/tbJjg6G3P8
@QUTRobotics

[EvalAI @eval_ai]

@eyshikaagarwal @facebookai Hi @eyshikaagarwal ,
Thank you for reporting this. We are working with the challenge organizers to fix it.

[Yash Kant at CVPR @yash2kant]

TextVQA (https://t.co/wrIrE2lHpL) and TextCaps (https://t.co/WAMDurpbFy) 2021 Challenges are live now!

Deadline: May 14th, 2021

Winners to be announced at #CVPR21 VQA Workshop (https://t.co/JkyEkbPmsJ).

1/n

[EvalAI @eval_ai]

@samarthbhargav Hi @samarthbhargav, Sorry for the inconvenience and thank you for reporting it. The issue is fixed now.

[Devi Parikh @deviparikh]

It's that time of the year again :) The 2021 VQA Challenge is now live! It closes May 7th.

Also note the TextVQA and TextCaps challenges that involve reading and reasoning about text in images.

[Dhruv Batra @DhruvBatra_]

It’s been a long journey. Thanks for the support GSOC @GoogleOSS!

[EvalAI @eval_ai]

We also have a few positions available for mentors who will help/mentor students during the summer. Please fill this form (https://t.co/BEXyVjzPdj) if you are interested in mentoring students in the @gsoc program on #EvalAI and we will reach out to you.

[EvalAI @eval_ai]

We are happy to announce that @project_cloudcv has been selected as a mentoring organization for @gsoc 2021. This is the 7th consecutive year and we are excited for the summer with #EvalAI project 🎉

[Google Open Source @GoogleOSS]

🥁 We have an announcement to make! 🥁

We are pleased to welcome 202 open source projects as our Google Summer of Code 2021 mentoring organizations, including 31 first-time organizations.

Read and see how to get involved ↓ https://t.co/YXgDTYdTEh

[EvalAI @eval_ai]

Here are the project ideas (https://t.co/dwg01ByNJS) for the summer and we are looking forward to student applications. We encourage students to engage with the community through our GitHub (https://t.co/HlXxIXnPmK) and Gitter (https://t.co/sxXAhgcD6M) channels.

[AI Habitat @ai_habitat]

Leaderboards of Habitat Autonomous Navigation Challenge 2021 are live!
Looking forward to your submission on PointNav w/o localization sensors (‘𝐺𝑜 5m 𝑛𝑜𝑟𝑡ℎ, 3m 𝑤𝑒𝑠𝑡’) and ObjectNav (‘𝑓𝑖𝑛𝑑 𝑎 𝑐ℎ𝑎𝑖𝑟’) tasks.

https://t.co/xDBR5r8C3R

[Abhishek Das @abhshkdz]

We've released a centralized evaluation server on @eval_ai to track community-wide progress on discovering electrocatalysts using ML.

https://t.co/j4nh8Idids
https://t.co/sdnSwtlhs9

(2/3)

[Dhruv Batra @DhruvBatra_]

13 challenges.
6 workshop organizers.
42 challenge organizers.
17 scientific advisors.
21 organizations.

1 workshop: https://t.co/0COJXOrkPZ #CVPR2021

[EvalAI @eval_ai]

MultiON Challenge 2021 - https://t.co/IRsE42e0ml

[EvalAI @eval_ai]

#EvalAI is the new home for Embodied AI challenges 🎉 This year we are hosting 5 challenges to be held at the Embodied AI workshop at CVPR 2021 from @facebookai, @StanfordSVL, @ai_habitat, @UTAustin, @IllinoisCS, @RoboticVisionAU, @robVisChallenge, @QUTRobotics

[EvalAI @eval_ai]

SoundSpaces Challenge - https://t.co/lE5U7AfFQj