Ismail Labiad

[Charles Arnal @arnal_charles]

🚨 AutoformBench: A Lean Formalization Benchmark for Advanced Mathematics

Our team at @AIatMeta is excited to release AutoformBench, a new formalization benchmark that pushes the Lean 4 coding capabilities of frontier models to their limits.

While SOTA models have become extremely competent at single-shot problem-solving, formalizing mathematics at scale and cost-effectively remains difficult. For instance, only about 42% of the results in openai/math could be formalized before release.

Furthermore, it is not clear and publicly known how much it costs to formalize big results, which is critical information if formalization is to become a part of the peer review process. This is the focus of AutoformBench.

The benchmark comprises 341 proving tasks drawn from diverse graduate- and research-level mathematical sources and formalized by human experts. Each task requires writing a substantial amount of auxiliary code, including lemmas, definitions, and more, placing the emphasis on models’ ability to build formal infrastructure rather than on mathematical cleverness alone.

The result is a demanding test: even the strongest models require tens of millions of tokens to complete 50% of the tasks (roughly $10 to $100 at current API rates).

Paper (arXiv coming soon): https://t.co/ZRr4BpKyCB
Code: https://t.co/gm3PDdpB04

Joint work by myself, Ahmad Rammal (@Ahmad3Rammal) and Vivien Cabannes, together with Niket Patel (@niketnpatel), Fabian Gloeckle (@FabianGloeckle), Bartosz Piotrowski, Amaury Hayat (@Amaury_Hayat), Remi Munos (@MunosRemi) and Julia Kempe (@KempeLab) from @AIatMeta, @NYUDataScience, and Ecole des Ponts. (1/6)

[Niket Patel @niketnpatel]

“An accumulation of facts is no more a science than a heap of stones is a house.” - Poincaré

LLMs are now proving results that have resisted mathematicians for decades. Finding interesting theorems without human guidance is a new bottleneck. We show that we can teach an LLM to do it!

TL;DR:
→ a quantitative notion of interestingness
→ 4.3× higher interestingness
→ a self-expanding discovery loop
[1/5]

[Ismail Labiad @is_labiad]

It was a pleasure working on this project with Matthieu Kowalski, Marc Schoenauer, @MunosRemi, and @KempeLab!
@AIatMeta @NYUDataScience

Check out the full paper for more: training details, compute analysis, ablations, examples, and theory.

https://t.co/GihBcffNpF

[Ismail Labiad @is_labiad]

Repeated sampling is the default way to scale LLM reasoning at test time. But token level noise often produces many near duplicate attempts that follow the same high level idea.
🧵 How can we cover more of the solution space without sacrificing throughput?

[Ismail Labiad @is_labiad]

At the same rollout budget, the trained small 7B concept generator roughly doubles the larger answer model pass@128 over naive repeated sampling, outperforms an untuned 32B concept generator, and transfers to an answer model from a different family it was never trained against.

[Mathurin Videau @mathuvu_]

Chinchilla writes loss as a sum of two independent terms: model size and data never interact. Kaplan's original law coupled them.
But interaction has an impact, dropping it bends predictions exactly where you extrapolate. Skaling, our new law, puts it back with one exponent. 1/8

[Krunoslav Lehman Pavasovic @KrunoLehman]

WaiT for the Signal: Simple Frequency-Aware Flow-Matching.

We show how to natively incorporate a fundamental property of images directly into diffusion models; setting a new pixel-space SOTA on ImageNet, while reducing compute.

📄 https://t.co/5ZJVOcfzQA

Full breakdown below👇

[Swann Bessa @swann_bessa]

🚨Introducing Cluster with Auctions (CwA), my first work at Meta Superintelligence Labs !

What if the vectors in your database had to bid at auction for a seat in their cluster? 🔨
That's CwA — and it makes partition-based vector search up to 4× faster out-of-distribution, using as many parameters as K-Means centroids.

With @hjegou, Matthijs Douze, @pierrefdz, Gergely Szilvasy

[Loic cabannes @loiccabannes]

Introducing Sparse Delta Memory (SDM) - The first work of my PhD 🎓.

SDM combines the GatedDeltaNet update with Product Key sparsity, enabling a recurrent state 3000x larger at the same FLOPs and significantly improving long-context performance.

Let RNNs be sparse!

[François Porcher @foporcher]

1/ My first PhD paper is out! 🎓

Title: Flow Matching in Feature Space for Stochastic World Modeling

tldr: we build stochastic world models directly in high-dimensional DINOv3 feature space, instead of relying on low-dimensional VAE latents.

[Juliette Decugis @DecugisJuliette]

What advantage to use, and when? Everyone's proposing new advantage functions for RL with LLMs but nobody knows why they work or fail.

We break this down and build FADE a self-adapting advantage to get +14% on LiveCodeBench v6 in 40% less steps.

Paper: https://t.co/fAX16uYPpt

[LucyZ @LucyZ47712090]

🚀 Brain2Qwerty v1 is in Nature Neuroscience, and today we're releasing v2 😎!
🧠⌨️ Same goal, but v2 slashes the best subject Word Error Rate from 52% to 23%, avg 39%—significantly narrowing the performance gap with invasive BCIs.
📄 Meta AI Blog: https://t.co/UXUZKIDniT 👇

[Elouan Gardès @elouangardes]

What if the metadata from acquisition was enough? 👀

Sometimes the most useful labels aren't labels at all.

Introducing📄 Who Needs Labels? Adapting Vision Foundation Models With the Metadata You Already Have

[Charles Arnal @arnal_charles]

Our team at @AIatMeta is excited to announce ATLAS: one of the largest automated formalization efforts to date.

ATLAS contains Lean 4 formalizations of both statements and proofs from 25+ mathematics textbooks, spanning dozens of domains, for a total of 500k lines of code. We are also releasing a flexible formalization harness and a companion paper.

External contributions are welcome!

Joint work spearheaded by our amazing PhD student Ahmad Rammal (@Ahmad3Rammal), together with Niket Patel (@niketnpatel ), Fabian Gloeckle (@FabianGloeckle), Amaury Hayat (@Amaury_Hayat), Remi Munos (@MunosRemi), Julia Kempe (@KempeLab), Vivien Cabannes, and myself from @AIatMeta, @NYUDataScience , and Ecole des Ponts. This is an ongoing effort; more details in the thread below.
(1/9)

[Tom Sander (Ph.D.) @RednasTom]

1/9 Excited to share TextSeal, our new state-of-the-art watermark for large language models at FAIR / Meta Superintelligence Labs (@AIatMeta) 🔐

Paper: https://t.co/XMJ1nYrEQF
Code: https://t.co/PAZygG69me

[Charles Arnal @arnal_charles]

(1/9) Experience replay can cut LLM RL training compute by up to ~40% (without hurting final accuracy—and sometimes improving it).

Paper: https://t.co/6YcAd6EBSy

[João Maria Janeiro @JoaoMJaneiro]

Happy to announce that today we released OmniSONAR (https://t.co/C8Lh98KF0q) and OmniMT.

In OmniSONAR, we have been able to really push the edge on largely mutlilingual embedding models, where representations across all languages are aligned like never before!

🧵1/n

[Belen Alastruey @b_alastruey]

Happy to share 🌍Omnilingual Machine Translation🌍

In this work @AIatMeta we explore translation systems supporting 1,600+ languages. We show how our models (1B to 8B) can outperform baselines of up to 70B while having much larger language coverage.

📄:https://t.co/isvEzRZbnw