Hierarchical LLM Reasoning
ReasonFlux is a hierarchical reasoning framework for LLMs that optimizes complex problem-solving using scaling thought templates.
It outperforms state-of-the-art models in mathematical reasoning.
Key contributions include:
• Structured Thought Template Library – A curated set of 500+ high-level templates designed to generalize across complex problems. This enables efficient retrieval and structured reasoning without exhaustive search.
• Hierarchical Reinforcement Learning – Instead of training on long Chain-of-Thought (CoT) sequences, ReasonFlux optimizes trajectories of thought templates, making problem decomposition more efficient.
• Adaptive Inference Scaling – The system dynamically scales templates during inference, adjusting reasoning depth as needed for different problems. This improves accuracy while reducing compute cost.
ReasonFlux-32B achieves 91.2% on the MATH benchmark, +6.7% over OpenAI o1-preview.
On AIME 2024, it solves 56.7% of problems, outperforming o1-preview by +27% and DeepSeek-V3 by +45%.
It also generalizes well on other competition-level datasets (e.g., OlympiadBench, Gaokao 2023).
This paper provides a closer look at the memory capabilities of LLMs.
It uses the Universal Approximation Theorem to explain the memory mechanism of LLMs. It also proposes a new approach to evaluate LLM performance by comparing the memory capacities of different models.
The Transformer architecture "functions as a dynamic fitting UAT model, with a strong ability to adaptively fit inputs. As a result, LLMs can recall entire content based on minimal input information. Since this memory can only be confirmed when triggered by input, we refer to it as ”Schrodinger’s memory.”
🔥 We are excited to announce the release of the next generation of open code LLM, Qwen2.5-Coder, and officially rename CodeQwen to Qwen-Coder. Yes, this is what everyone has been waiting for!
HF Demo: https://t.co/k8ZAW5JzdC
Technical Report: https://t.co/SrUKYUU3kE
🧵1/7
Can OpenAI o1 tackle hard reasoning problems? We tested it on the complex reasoning task in our Grokked Transformers paper. It turns out that o1-preview also struggles a lot like earlier LLMs; on the other hand, a grokked transformer can nail it near-perfectly.
@instablog9ja It appears that Nigerians and Nigerian bloggers have an enduring penchant for bringing this up on their page every Ramadan season. Perhaps one day, y’all will elaborate on your fixation with gay contents.
@IamEkene_@instablog9ja You don’t even know them enough to know that they didn’t adopt their kids, yet you feel strongly inclined to form an opinion about them. Rest!
@instablog9ja Not in the UK, but I believe you’re wrong. A person earning 51K would be subject to the same tax % as someone earning 49K on the initial amount, any income beyond that threshold (e.g. 49K) could incur a higher tax rate in a progressive tax system, which would on the 2K left.
@semiotec_ @pandathecoder It’s because computer handles FPN. The computer uses base_2 floating point number, and humans use base_10. It’s the same result you’ll get across multiple programming languages.
Machine Learning Formulas Explained! 👨🏫
This is the formula for the Binary Cross Entropy Loss. This loss function is commonly used for binary classification problems.
It may look super confusing, but I promise you that it is actually quite simple!
Let's go step by step 👇