We release the first improvement to the lower bound on the unrestricted circuit complexity of 3-SAT. Central to P vs NP, 3-SAT is arguably the most intensely studied and foundational problem in theoretical computer science and one of the most for mathematics. Yet in the 50+ years since 3-SAT was defined, there has been not a single improvement to the lower bound on complexity of this problem. If you can improve the lower bound on 3-SAT enough, then you can prove P≠NP and resolve the Millennium Prize Problem. This is the first such improvement to the lower bound on 3-SAT in its history.
This is not to be confused with the upper bound on 3-SAT, which has seen consistent progress leading up to OpenAI's seminal work reducing the upper bound sufficiently to disprove the Exponential Time Hypothesis (ETH). However lower bounds have seen virtually no progress before our work.
Sanjeev Arora, a Princeton professor, and two-time Gödel Prize recipient, and Boaz Barak, a Harvard professor and ACM Fellow wrote in their complexity textbook: “Progress on general circuits has been almost nonexistent”. Lance Fortnow, an ACM Fellow wrote that “circuit lower bounds hit a brick wall”. Their expertise and textbooks, research and community leadership make these assessments useful historical context for why even improvements to linear lower bounds receive attention.
Our work has been formally verified in Lean.
Link: https://t.co/D6BvDcWaRc
@mert_gulsun Is it cuz Astra, Fastino GLiDE and Sol are loopy bois that scale latent space reasoning perhaps by any chance? 🤭 (Latent space reasoning ofc being orders of magnitude more efficient)
Fine-tuning a small decision model can reduce its classification errors by >50%.
In a recent study, AI One experimented with training a 9B open weight decision model, customized to handle 50 different enterprise workflows.
Using roughly 5,000 synthetic training examples, they increased the model's accuracy from 92.55% to 96.56%, cutting classification errors by more than half and reaching the performance range of the strongest frontier configuration they tested.
Training decision models on domain-specific data is something we've been working to make easier at @fastinoAI.
Our family of open weight decision models, GLiDE (0.4B-27B), is built for fast, accurate decision-making, ranking #1 on the Hugging Face Decision Index.
With the Fastino Training Agent, you can customize GLiDE for your own workflows, from research and synthetic data generation to training and evaluation, without manual tuning.
Send me a message if you're interested in fine-tuning your own open weight models.
“GLiDE stands out on both counts: it achieves the highest score among all System One models and sits on our Pareto frontiers for both latency and option-order stability.”
- System One Arena, an independent third party evaluator's thread on Hugging Face regarding our Decision Model.
Gemma 31b with enough post training can always beat qwen 27B. For a long time the best base for open source decision models on the HF decision index was Qwen 27b but now the top paradigm is dominated by Gemma 31b. Google makes great models.
The question is do the American tech companies have the incentive to open source their models?
I guess it’s a strategic sabotage of competitors like anthropic and OpenAI when you don’t have the same level of expertise and resources to train a Fable 5.2 or Opus 5.5. This is game theoretically optimal
@natolambert But do they have the incentive to do so? I guess they do but it’s harder for American companies to get away with distillation which is a core part of the open source formula
Meta found a better version of something as fundamental as k-means?? (Sidenote: this is a bull argument for ai given we’re seeing genuine increased productivity in economically important areas)
Sorting huge amounts of data into groups, a basic step behind search and curating AI training data, now takes about a tenth of the compute.
The classic method, k-means, needs about 25 passes over the data. Meta FAIR's e-means gets the same result in about 2.
It updates each group's centre a little at a time from small random samples. Like every version of k-means, it has a quiet failure: some groups stop getting any data and sit idle. In one test without a fix, 26% of the groups ended up idle.
The fix is to split a busy group in two every few steps and hand half to the idle one. Idle groups fell to 0.3%, and the paper says this rule mattered more than any other part of the recipe.
So the same grouping job finishes about 10 times faster on the same machines.
In 1966, Paul Cohen won the Fields medal for showing the independence of: the continuum hypothesis and the axiom of choice.
Gödel reacted to Cohen’s Fields medal by saying that he had already proved the result in the 1940s. Yet when pushed for a proof by his peers, Gödel was unable to reconstruct one from his notes.
"The first task von Plato set Claude was to discover if Gödel’s workbooks had a proof of the independence of the axiom of choice, which would reveal that Gödel had been telling the truth in 1966.
The answer came back: yes, almost. “He was just one step away,”"
Tuning how an AI behaves usually means more training. Meta FAIR showed you can adjust it after training with basic math on the weights.
They took a loosely graded coding model and a strict one. The weight difference between them acts like a dial. Pushing one step past the strict model made its code 1.30x faster on slow problems (over 1 second), and faster 71% of the time, with zero extra training. It made more mistakes.
You can push past the strictest model you trained along that same line, instead of retraining for a stricter trade-off.
I fine-tuned 2 GLiDE models for a few dollars on previous festival line up posters and my Spotify listening history - I can now drag and drop new line ups as they drop and see if it fits with my taste and quickly generate playlists.
Try our training API today: https://t.co/JvQa4JUz8a
Technical Thread below
Technical report: "Recurrent Looped Transformers: Pushing the Frontier in Decision Models and Scaling Reasoning in Latent Space" (Fastino Labs). It covers the four trained-loop attempts that failed, the 16 alternative loop shapes that all scored lower, and results for every model size.
https://t.co/Wz5h4NNRM9
We ran the middle of a finished model twice before it answered, changed NONE of its weights, and it got better.
In our 27B frontier model, which is a decision model, we ran two bands of middle layers a second time each. We used no new parameters and wrote no extra text.
On held-out questions it gained 0.56 points. It took 112 ms per answer vs the 83 without looping.
Across five model size 0.4-27B the loop did nothing below 1B parameters and was the clearest gain at 27B. Our 9B gained nothing until another training round made it stronger. A third loop pass on the neural network layers helped some tasks but cost 3 points on a causal-reasoning test. Training the loop in did worse than looping the finished model.
What an honour it is to be in a frontier model competition with the legendary Google DeepMind and OpenAI.
btw Fastino GLiDE is also ranked # 1 on Artificial Analysis and Hugging Face.
Below is a comparison table of Gemini Decision 1 vs Fastino GLiDE.
Introducing Gemini Decision 1, our new frontier decision model in the Gemini API.
It is $0.10 per 1M input tokens with no additional cost. With support for image, audio, video, and PDF natively, 256K context window, & 1K questions.
You can try it for free in the API!