Mithil Vakde

Trying to solve sample efficiency

prev: Engineering physics @iitbombay '23

[Mithil Vakde @evilmathkid]

I got 37% on ARC-AGI-2 Kaggle without LLMs!

100M looped transformer trained from scratch

- Likely highest score without an LLM
- Peaked at 6th place
- Beats last year's winning LLM
- Looping, added in the last week: 33% → 35% → 37%

thanks to @arcprize @fchollet @mikeknoop and @GregKamradt for the competition!

Link: https://t.co/iDBUrHhmdI
(writeup for the paper track)

[Mithil Vakde @evilmathkid]

@yacinelearning @arcprize @fchollet @mikeknoop i think a lot more can be done with looping!

[Mithil Vakde @evilmathkid]

I got 37% on ARC-AGI-2 Kaggle without LLMs!

100M looped transformer trained from scratch

- Likely highest score without an LLM
- Peaked at 6th place
- Beats last year's winning LLM
- Looping, added in the last week: 33% → 35% → 37%

thanks to @arcprize @fchollet @mikeknoop and @GregKamradt for the competition!

Link: https://t.co/iDBUrHhmdI
(writeup for the paper track)

[Mithil Vakde @evilmathkid]

@misbahspeaks @arcprize @fchollet @mikeknoop Thanks Misbah!

[Mithil Vakde @evilmathkid]

@NirantK @arcprize @fchollet @mikeknoop Thanks Nirant!

[roon @tszzl]

we should not like a future where there are 1 to 2 AIs and the rest are illegal. we should want a cambrian explosion of the types and possibilities of minds, experiencing new joys and thinking new thoughts. ensuring that this is compatible with human flourishing is only step 1

[Mithil Vakde @evilmathkid]

@datavorous_ @arcprize @fchollet @mikeknoop shhh havent yet launched it

[Anshul Bhide @anshulbhide]

LLMs require very large amounts of data and arguably the path to ASI runs through a different model architecture that’s more data sparse. Amazing what @evilmathkid has done with compute constraints on ARC-AGI-2.

cc @fchollet

[Mithil Vakde @evilmathkid]

@willdepue @arcprize @fchollet @mikeknoop thanks will!

[Mithil Vakde @evilmathkid]

@PariharCodes @arcprize @fchollet @mikeknoop ??

[Mithil Vakde @evilmathkid]

@shyamalanadkat @arcprize @fchollet @mikeknoop thanks shyamal !

[Mithil Vakde @evilmathkid]

@BhatiGarvin @arcprize @fchollet @mikeknoop Estimate it took 6 hours on Kaggle (4xL4s)

Training should take about 5hrs
Decode less than an hour

kaggle had really long queues, so cant know for sure

[Mithil Vakde @evilmathkid]

@anindyadeeps haha thanks!

[Mithil Vakde @evilmathkid]

@xennygrimmato_ @arcprize @fchollet @mikeknoop thanks!

[Anindyadeep @anindyadeeps]

And Mithil strikes again, this time casually beating opus

[Mithil Vakde @evilmathkid]

@bryanmusuku @arcprize @fchollet @mikeknoop working on this :)

[Mithil Vakde @evilmathkid]

@ram_devv @arcprize @fchollet @mikeknoop wdym?

alternative attention impl?

[Mithil Vakde @evilmathkid]

@AashaySachdeva @arcprize @fchollet @mikeknoop its extraordinary how well it works

i had basically given up, looping was a hail mary and boom it just worked

[N8 Programs ⏹️🤝⬆️ @N8Programs]

THE GOAT (of resource efficient arc impls) RETURNS

[Mithil Vakde @evilmathkid]

@82deutschmark @arcprize @fchollet @mikeknoop Haha I sweating bullets because of the submission problems on Kaggle

[Mithil Vakde @evilmathkid]

@brycesandlund @arcprize @fchollet @mikeknoop thanks!

[Mithil Vakde @evilmathkid]

@GregKamradt Thanks!

Less than 4 hrs on the 240 placeholder tasks on Kaggle

The hidden set is larger so I expect around 5 hours