Sammy Atman

ai researcher, ex-faang, unstoppable.

[Sammy Atman @SammieAtman]

Our project to run DOOM on LLMs won 🥇 place in the finetuning track the @MistralAI hackathon.

Yes, there are better approaches you could use to train a game playing agent, but we just HAD to find out if Mistral-7B could run DOOM.

More details on what we did..

[Sammy Atman @SammieAtman]

Videos from a moving camera are more truthful and higher entropy than text so you'd expect superior zero-shot performance at the limit. Provided the proliferation of video generated content doesn't exceed the yield of real video data

[Aran Komatsuzaki @arankomatsuzaki]

Veo 3 = Zero-shot video reasoner

• Trained on web-scale video, shows broad zero-shot skills (perception → physics → manipulation → reasoning)
• New “Chain-of-Frames” reasoning = visual analogue of CoT
• Big jump Veo2 → Veo3: edits, memory, symmetry, mazes, analogies
• Caveat: still lags SOTA on depth/physics; cost high

[Sammy Atman @SammieAtman]

@SakanaAILabs Discovery is the ultimate long-horizon task, this is an important framing of continual learning

[Sammy Atman @SammieAtman]

@adityaag Zero-shot learning is an existence proof of this idea. In some ways, the mechanisms we associate with general intelligence already exist to some extent (even if the model's knowledge distribution has gaps)

[Sammy Atman @SammieAtman]

Great designers simplify form while retaining function: min ||f(z) - f(X)|| + ||z||

Great artists create forms that backproject into complex ideas: min ||X-f^-1(z)||

[Sammy Atman @SammieAtman]

Many people conflate intelligence and knowledge when discussing AGI

[Sammy Atman @SammieAtman]

@eladgil Hysteresis by induction

[Sammy Atman @SammieAtman]

The interesting problems are being solved with vision (and control data) right now. LiDAR and depth can improve on these but we don’t necessarily need it to make progress on most environments in the short term

[Chris Paxton @chris_j_paxton]

Fundamentally this is one of the most interesting discussions in robotics to me right now: is machine learning good enough that we can do away with single purpose sensing like lidars and depth sensors, and with more precise and high quality models? The first place we will really see this competition play out in a dramatic way is self driving cars

[Sammy Atman @SammieAtman]

The Universal Approximation Theorem is greatly misunderstood.

It makes a claim about the expressiveness of a network but not optimizability. As compute scales, the learning efficiency of the algorithm becomes the limiting factor

[Jack Morris @jxmnop]

this is not true at all. in fact, a main point of the OpenAI scaling law paper was exactly the opposite:

only *with* compute does it become clear that Transformers are better than LSTMs

[Sammy Atman @SammieAtman]

RL : NN :: tech : society
pretraining -> effective exploration -> maximizes long-horizon reward

[Sammy Atman @SammieAtman]

@gdb It's the best

[Sammy Atman @SammieAtman]

@rohanpaul_ai @GoogleDeepMind Biggest takeaway is the sheer dominance of BM25 over time

[Sammy Atman @SammieAtman]

@eshear its a good post sir

[Sammy Atman @SammieAtman]

@julianweisser They already assumed they could by asking the question

[Sammy Atman @SammieAtman]

Thinking is the process of drawing structure out of an amorphous model of reality

[Sammy Atman @SammieAtman]

@timholtski @elonmusk Ok, what's one compelling reason a person has to be physically present in the tower?

Also most of the necessary data is aggregated through STARS already and most of the comms consist of routine stuff.

Emergencies can be handled by a human listening at a remote location.

[Sammy Atman @SammieAtman]

@nrehiew_ Need capsule networks for some of these

[Sammy Atman @SammieAtman]

The real psyop isn't sandbagging about compute costs, its inflating test time compute costs that aren't amortized over time