@yx3st ah loop models do really well on algorithmic reasoning tasks and we want to offer an alternative that does better. Maybe our wording for the tweet was a bit too aggressive ๐
Bye bye looped reasoningโฆ and hello Energy-Based Reasoners (EBRs)โก!
Your looped reasoning model would be better off as an energy-based model (EBM)...
TLDR:
EBRs uses 345x less inference compute and 2.6x fewer training FLOPs than the best baselines
EBRs are 7x-43x more sample efficient than the best looped baselines at inference.
EBRs achieve 99.9% on Sudoku-Extreme
EBRs achieve near 100% accuracy on multi-solution benchmarks
๐งตThread:
@lwilczoch@AlexiGlad@Ben_Hoov I think both EBR and CRAM can provide really exciting insights for EBMs; and hopefully we can leverage that to scale EBMs to bigger problems :)
Also check out @Ben_Hoov og Energy Transformer work! Learned lots from it
Really cool work from @Ben_Hoov. The goal for our paper was to see if framing looping as energy minimization can generalize as well as models like TRM, so we use less inductive biases than CRAM. The goal of their paper was to show that really tiny models parameterized under certain priors and constraints can learn these puzzles a lot more efficiently
@QuantumTumbler Thanks for the question! I completely agree! The benefit with training a verifier / energy is that they have been shown to be robust to ood shifts as long as the structure of the problem is still be verifiable
@AdamKleinML@AlexiGlad Co-author of the paper here. Cool suggestion! this is definitely worth trying along with other design choices! If one still wants to keep the descent properties of EBMs then I think it could work with a PSD matrix as the preconditioning
@QuantumTumbler Ah that's a good question. For EBMs this stopping threshold means the model assigns a stable point to what it thinks are good solutions. We haven't tested this for ood constraint structures or unseen sizes, but as long as the energy verifier remains meaningful then it could work
@QuantumTumbler Thanks for the question! Most (if not all) of the benchmarks we used have test sets that are not part of training data. As for false stop rates, check out Table C.2 in the appendix of our paper! We perfectly captured the accuracy that a full fixed-budget iteration obtains
Energy-Based Reasoners: Your Recurrent Reasoner Should Be an Energy-Based Model
By using gradient descent on learned energy to guide looped reasoning, Energy-Based Reasoners hit 99.9% Sudoku-Extreme accuracy with 345x less inference compute than rivals.
https://t.co/ZHJcyI7GdB
[7/N] We believe that reasoning works better as verification than as step-by-step updates.
Energy is an elegant way to do iterative reasoning by unifying updates, stopping, and the answer selection under one objective.
More works coming in LLM reasoning and world-model planning!
And weโre working on ARC-AGI results, coming soon ๐
๐ Paper: https://t.co/0OCUQ77FqQ
๐ AlphaXiv: https://t.co/oC6HwhFGSC
๐ HF paper: https://t.co/oxJaYv4J9q
๐ Project: https://t.co/kHsWginqTs
๐ป Code: https://t.co/ol7UyGZTse
w/ @AlexiGlad, @MichaelLi142465, @hengjinlp, @du_yilun
[6/N] ๐ On N-Queens & Graph Coloring, EBRs beat every baseline, with near-perfect accuracy from one sample and up to 1.6ร more distinct solutions.
@WangYw251@XiaoyuW9401147 Thanks for compiling these! every work I read seems to just get better FID / better looking images ๐ญ so isolating the method from the bag of tricks is quite important