Max Y

Systems engineer that cosplays researcher at night.
Main interests: architectures, interpretability, reasoning

[Max Y @waxhn]

(1/6) Happy to share a project I've been working on! Even though things didn't work out, here's what we did and learned:
Huginn-3.5B was released in Feb. It's one of the first LLMs that can utilize varying amounts of compute per token. We set out to optimize this compute usage.

[Max Y @waxhn]

@reza_byt @lucia_quirke (I just realized, huge fan of MoR btw 😂)

[Max Y @waxhn]

@lucia_quirke @reza_byt Tbh I don’t have an opinion. But re-reading Reza’s reply, I think he’s referring to the numerous loop transformer literature that existed long before TRM instead. E.g. UT, huginn, MoEUT, etc.

[Max Y @waxhn]

@lucia_quirke @reza_byt I fell for this as well lol

[Mithil Vakde @evilmathkid]

@waxhn No, I am not talking about the looping. TRM separately has 435M trained embedding parameters.

For each task, you loop over the 7M parameters, with a separate embedding parameter used for each augmentation of the task

https://t.co/sKTQ45ltzz

[Max Y @waxhn]

@Grad62304977 Yeah to add on, dynamic looping doesn’t really work since attention between Q/KVs from different loops sacrifices a lot. It still somewhat works out of the box (due to the loop structure), which makes it even more misleading

https://t.co/OuXvauRvZq

[Max Y @waxhn]

(3/6) While this approach was decently elegant and achieved ok training results, we later discovered the entire approach was flawed.

Adaptive compute requires mixed-depth attention, since during inference we would be missing latents of prior tokens in the sequence that early exited. We thought Huginn could handle mixed depth attention, but the penalty was much bigger than expected.

In hindsight, this is an example of “you get what you trained for”. While Huginn is trained to run with any number of loops, due to how AR language models are pretrained, it was only trained for “uniform depth” - all tokens in each sequence ran for the same number of loops.

(The Recurrent Depth Transformer architecture of Huginn still ensured mixed depth attention works to some extend. But the penalty was higher than expected.)

[Susan Zhang @suchenzang]

watch all the accounts rn who have absolutely no clue what the difference between "model architecture" (latent space manipulation) vs "thinking traces" (model output) actually are

[Max Y @waxhn]

@amir A bit confused, loop transformers don’t inherently hide CoT. Unless they’re also doing some other latent method; but that’s orthogonal to recurrent depths?

[Max Y @waxhn]

@eliebakouch One factor is how MoEs interact with recurrence. Theoretically, looping nd choose k experts d times is same compute+params but more expressive than n choose k experts with d layers.

Adaptive compute however requires attention quality tradeoffs: https://t.co/YbOBdmnmTo

[Max Y @waxhn]

(4/6) In conclusion, the biggest learning is that for a recurrent depth transformer to utilize adaptive compute, mixed depth attention has to be accounted for during training.
E.g. the new Mixture of Recursions by @raymin0223 seems to have addressed this via top-k expert routing

[Max Y @waxhn]

Paper link: https://t.co/IeCT00kwP0

[Max Y @waxhn]

(1/6) I’ve built the strongest general-purpose Competitive Pokémon AI to date.

PokaiTrainer uses decision-time equilibrium search (ReBeL & SoG), and can pilot a wide range of teams on the live Showdown Bo3 ladder.

With a 59%wr over 150 sets, it briefly reached top 500🧵

[Max Y @waxhn]

(sry to clarify, VGC specifically)

[Max Y @waxhn]

(6/6) Overall I had a blast working on PokaiTrainer, and am incredibly proud of the results!
paper link: https://t.co/IeCT00kwP0
And if anyone wants to test it out yourself: https://t.co/OJkP0ge8eu

[Max Y @waxhn]

Proud to share the first Pokemon VGC AI that can battle at near expert level. For until Sunday, it'll be hosted on Pokemon Showdown, available for competitors to use as a practice partner! https://t.co/qUWtgsnZhK

[Max Y @waxhn]

(5/6) Training was done in rounds of self-play datagen then train on solved CFR values. At roughly 250*(L40S +48CPU) hours, PokaiTrainer is able to compete on the Showdown ladder, maintaining a steady 1350-1400 elo band, at its peak reaching 1492 elo and top 500. Across 150 sets, it played by randomly picking teams for each match from a diverse set of archetypes, including the notoriously difficult perish trap strategy.

And interestingly, while I desperately tried to save compute, our network seemingly plateaued at ~3 orders of magnitude less compute than ReBeL/SoG/AlphaGo. I do not have a good explanation.

[Max Y @waxhn]

@evilmathkid Oh damn was not aware. Yikes…

[Max Y @waxhn]

I'll be leaving this up for the time being, since I don't have any use of my host machine for a while. But future availability is not guaranteed. For those interested, feel free to message me!

[Max Y @waxhn]

Proud to share the first Pokemon VGC AI that can battle at near expert level. For until Sunday, it'll be hosted on Pokemon Showdown, available for competitors to use as a practice partner! https://t.co/qUWtgsnZhK

[Max Y @waxhn]

This AI should serve as a pretty decent practice partner for worlds competitors! When setting up the battle, you can give it a pokepaste of what team you want it to pilot.
#pokemonworlds #vgc #pokemonㅤㅤㅤ

[Max Y @waxhn]

Proud to share the first Pokemon VGC AI that can battle at near expert level. For until Sunday, it'll be hosted on Pokemon Showdown, available for competitors to use as a practice partner! https://t.co/qUWtgsnZhK