Mithil Vakde

Sample efficiency research.

prev: Engineering physics @iitbombay '23

[Mithil Vakde @evilmathkid]

44% on ARC-AGI-1 in 67 cents!
Trained from scratch in 2hrs on a 5090

Matches TRM, beats HRM and is way faster & cheaper
No recursion, just a transformer

Also, 7% on ARC-2 🧵

[Mithil Vakde @evilmathkid]

@ergodicthought @CSProfKGD @ylecun Actually lecun says exactly that in the slide before this

I assume the subtext is "for human level ai", build these types of world models.

[Mithil Vakde @evilmathkid]

@ergodicthought @CSProfKGD @ylecun irl in a separate interaction I mean

also found a video of this talk. He says "world models should be ..." https://t.co/UAd9uQJR0h

[Mithil Vakde @evilmathkid]

@ergodicthought @CSProfKGD @ylecun I think he is just saying "build world models with these characteristics". Thats how it came across irl and in his paper

Idk why Kosta called it "his definition". Im pretty sure Lecun knows they are all models of the world

[Mithil Vakde @evilmathkid]

@ergodicthought @CSProfKGD @ylecun i think the prediction space in those 2 are designed and aim for full fidelity while lecun wants to predict representations that are learnt and intentionally lossy (ignore details irrelevant to task)

[Adithya Dsilva @AdithyaDsilva]

Hiring IC+ level computer engineers with an interest in building large scale, dynamic scrapers that work autonomously.

In person @ indiranagar, Bangalore.

(IC - someone confident to the point of being the authoritative say in the team. Not bothered by age / exp)

[Mithil Vakde @evilmathkid]

@o_v_shake friend needed it back

[Mithil Vakde @evilmathkid]

Anyone have a used mac i can buy? I dont have a laptop anymore

Older ones like M1/M2 also work

[Mithil Vakde @evilmathkid]

@mike64_t @willdepue @francoisfleuret what's SPD btw?
chatgpt is confused too

[Mithil Vakde @evilmathkid]

the most entertaining outcome strikes again

[Mithil Vakde @evilmathkid]

@gauravisnotme Nobody thinks we can catch up. Its worse
- Those who understand this is existential have given up
- Those who don't think AI is a gimmick

Capital exists but not a single serious effort has been made so it has gone to grifters and the incompetent

[Mithil Vakde @evilmathkid]

@plugyawn oh yeah
but i think vie doesn't think big enough
why only scaling or RL breakthroughs?

[Mithil Vakde @evilmathkid]

@pranesh @nileshtrivedi @NandanNilekani good job with the sneaky ad hominem. it distracts from the fact that your frame and understanding of my pos are both wrong

happy to continue the discussion if we stick to arguments

[Mithil Vakde @evilmathkid]

> preventing modification is not the goal of alignment
??? whats the point of alignment if i can modify and de-align it

> no amount of "alignment" can prevent modifications
today, partially true. but you assume that future models will look like the LLMs of today.

you are also making other assumptions that are unlikely like "open models are gonna be as capable frontier ones"

[Mithil Vakde @evilmathkid]

@pranesh @nileshtrivedi @NandanNilekani You sure we'll have capabilities to modify powerful aligned models a few years in the future?

One of the most important goals of alignment is to prevent modification

[Mithil Vakde @evilmathkid]

This is extremely short sighted. No reason why this will continue to be true.

Steelman: As alignment capabilities improve, frontier open models will be aligned to the home countries.

In this future indias entire tech stack will be rendered useless, forcing us to depend on foreign tech for everything
https://t.co/nQGoaIi6D9

[Mithil Vakde @evilmathkid]

This argument and Sutton's Big world hypothesis is why I'm very optimistic about humanity's future

Bring on the all-encompassing automators. We'll still find new things to do

[Dwarkesh Patel @dwarkesh_sp]

.@michael_nielsen made an incredibly interesting argument about why the tech tree is actually way larger than we realize, and how our descendants will get to explore very little of it.

[Mithil Vakde @evilmathkid]

finally got some time to try openai's parameter golf.
only 2 days left tho and the top scores are extremely optimised

lets see if i can work some magic

[Mithil Vakde @evilmathkid]

Reduce kolmogorov complexity in a ~turing machine defined by 8xH100s, PTX, a CPU and 10min while also optimising the hell out of the code execution

Love it

[Mithil Vakde @evilmathkid]

Feeling FOMO over all the cool work on optimisers.

How hard can the linal be? was my fav course in 1st year

[Mithil Vakde @evilmathkid]

@pfau @leothecurious Then the award loses meaning