Small recurrent transformers like TRM and HRM have shown great promise. We introduce the Lattice Deduction Transformer, a recurrent transformer grounded in abstract interpretation. Using symbolic lattice structures, the model learns to approximate sound logical deduction.
Introducing Lattice Deduction Transformers: An 800k-parameter looped transformer that reasons like a SAT solver achieves 100% on Sudoku-Extreme with only 15 minutes of training.
A collaboration between @axiommathai, @AmherstCollege and @BarnardCollege.
A little less than 24 hours after OpenAI's release of 722 new results, most of which I do not have the background to understand, I am still in utter shock. For now, I'm trying to make sense of the variance of random k-SAT hitting time result.
Here's a sneak peek of some in progress work on the Lattice Deduction Transformer. Below is the solve replay of ARC 1 task 405, where the model is executing DPLL-style search with learned deduction and conflict detection.
arXiv has updated our policy on rate limiting for all submitters.
This update was made to fairly distribute moderator time & support the arXiv community of staff, volunteers, readers & authors.
Please read our announcement to learn more: https://t.co/lvzWuiv0hx
@arbdwj My course project I put on arxiv got cited by 2 ICML papers, I now never underestimate the ability of LLMs to find arxiv papers that are otherwise unknown.
Lattice Deduction Transformers has been accepted at @NeurIPSConf! 🎉
We're excited to present this work and grateful to everyone who has engaged with it so far.
Here's a glimpse of LDT in action and how it works. We’re also preparing an improved revision of the paper, more soon!