Manjunath

[Sebastian Raschka @rasbt]

Following up on my reasoning model article, I just read the new "s1: Simple Test-Time Scaling" paper, which describes an interesting method for improving reasoning models using a combination of pure supervised finetuning (SFT) and scaling inference compute.
In short, their approach is 2fold:

1. Create a curated SFT dataset with 1k examples that include reasoning traces.

2. Control the length of responses by:
a) Appending "Wait" tokens in certain cases to get the LLM to generate longer responses, self-verify, and correct itself, or
b) Stopping generation by adding an end-of-thinking token delimiter (“Final Answer:”). They call this length control "budget forcing."

Budget forcing can be seen as a sequential inference scaling technique because it still generates one token at a time (but just more of it). In contrast, we have parallel techniques like majority voting, which aggregate multiple independent completions.

They found their budget-forcing method to be more effective than other inference-scaling techniques I’ve talked about, like majority voting.
If there's something to criticize or improve: I would’ve liked to see results for more sophisticated parallel inference-scaling methods, like beam search, lookahead search, or the best compute-optimal search described in Google’s "Scaling LLM Test-Time Compute Optimally Can Be More Effective Than Scaling Model Parameters" paper last year. Or, even a simple comparison with a classic sequential method like chain-of-thought prompting ("Think step by step").

Anyway, it’s a really interesting paper and approach!

Bonus: Why "Wait" tokens? My guess is they were inspired by the "Aha moment" figure in the DeepSeek-R1 paper, where researchers saw LLMs coming up with something like "Wait, wait. Wait. That’s an aha moment I can flag here." which showed that pure reinforcement learning can induce reasoning behavior in LLMs.
Interestingly, they also tried other tokens like "Hmm" but found that "Wait" performed slightly better.

[Ai2 @allen_ai]

Here is Tülu 3 405B 🐫 our open-source post-training model that surpasses the performance of DeepSeek-V3! The last member of the Tülu 3 family demonstrates that our recipe, which includes Reinforcement Learning from Verifiable Rewards (RVLR) scales to 405B - with performance on par with GPT-4o, and surpassing prior open-weight post-trained models of the same size including Llama 3.1

[Jiayi Pan @jiayi_pirate]

We reproduced DeepSeek R1-Zero in the CountDown game, and it just works

Through RL, the 3B base LM develops self-verification and search abilities all on its own

You can experience the Ahah moment yourself for < $30
Code: https://t.co/UcGKN2SVGj

Here's what we learned 🧵

[Philipp Schmid @_philschmid]

For those trying to understand @deepseek_ai Group Relative Policy Optimization (GRPO). Here, in simple steps:

1️⃣ Generate multiple outputs for each prompt using the current policy
2️⃣ Score these outputs using a reward model (rule or outcome)
3️⃣ Average the rewards and use it as a baseline to compute the advantages
4️⃣ Update the Policy to maximize the GRPO objective, which includes the advantages and a KL term

[Sayak Paul @RisingSayak]

Introducing a high-quality open-preference dataset to further this line of research for image generation.

Despite being such an inseparable component for modern image generation, open preference datasets are a rarity!

So, we decided to work on one with the community!

[Yuchen Jin @Yuchenj_UW]

The highest-scored paper at ICLR 2025 with full scores, 10, 10, 10, 10! The first time in ICLR history?

IC-Light is designed to control image lighting. They managed to collect >10 million images for training illumination editing models, with amazing results on SDXL and Flux backbones.

It's from the same author as ControlNet and has already gotten 5.7k stars on GitHub. I love seeing such research with open-source code and real-world impact. AI is empowering artists to create incredible things.

[Devendra Chaplot @dchaplot]

We just released Pixtral 12B paper on Arxiv:
https://t.co/3ljqF7WtAt

[Pedro Domingos @pmddomingos]

Physicists think AI is physics.
Statisticians think AI is statistics.
Mathematicians think AI is mathematics.
Psychologists think AI is psychology.
Neuroscientists think AI is neuroscience.
And they’re all right.

[Qian Liu @sivil_taram]

📚Introduction to a new paper "Performance Law of Large Language Models"🤖

This paper presents a new empirical equation that directly predicts the performance (i.e., MMLU score) of LLMs by fitting a law on top of several hyper-parameters ⬇️. Leveraging❗️10 open-source models from 2024 as the train set, the law accurately predicts MMLU scores for a range of models – from tiny 0.5B to 1T parameters from 2020 to 2024 with different architectures (dense and MoE)!

🔬 The performance law uses a pool of hyper-parameters for fitting: 🧙‍♂️number of layers (N), hidden size (h), intermediate size (d), training data size in trillion tokens (T) and model size in billion parameters (S). The equation also incorporates concepts such as a model saturation clip✂️ and an unstable discount factor 🎢 to account for limitations in model training and architecture.

For dense models, the MMLU score can be predicted using a log-linear regression function 📈🧮 that combines the above hyper-parameters as variables. For MoE models, the paper introduces an expansion factor that considers the number of activated parameters🎈.

🔍 Key insights:
🏗️ Deeper models are powerful but tricky to train
🧠 Hidden size can matter more than FFN size
🤖 Top AI companies likely use similar quality data
🔀 MoE models are promising but challenging

🚀 Applications:
The law can be a LLM development GPS, guiding architectural choices and optimizing resources. By continuously comparing your model's actual performance against the law's predictions, you can ensure your model is on the right track.

🕵️‍♂️Limitations:
The law isn't perfect. Currently it does not account for all factors, for example: the data quality, the data mixture (https://t.co/HAEfjIiwEf) and the vocabulary size (https://t.co/tMPkq2gYpF).

🥚Bonus:
Using the law, you can also try to infer the model size of close-source models. The authors took a shot at unmasking the mysterious GPT-4. Here's what they deduced:
1⃣Layers: 96
2⃣Hidden Size: 12,288
3⃣FFN Size: 49,152

What do you think? Does this align with your expectations for GPT-4's architecture? 🤔

[Gradio @Gradio]

🚀 Scribble SDXL ControlNet with Gradio ImageEditor component works like magic!

Check out the model and cool Spaces👇

[Philipp Schmid @_philschmid]

Llama 3 released! 🚨🔔@AIatMeta just released their best open LLM! 👑🚀 Llama 3 is the next iteration of Llama with a ~10% relative improvement to its predecessor! 🤯 Llama 3 comes in 2 different sizes 8B and 70B with a new extended tokenizer and commercially permissive license! ✅

Blog: https://t.co/VrceFpjI1o
Models: https://t.co/SdkOeWURuM

New and improvements to v2✨:
🔠 Trained on 15T Tokens & fine-tuned on 10M human annotated samples
🧮 8B & 70B versions as Instruct and Base
🚀 Llama 3 70B best open LLM on MMLU (> 80 🤯)
🧑🏻‍💻 Instruct good at coding 8B with 62.2 and 70B 81.7 on Human Eval
✍🏻 Tiktoken-based tokenizer with a 128k vocabulary
🪟 8192 default context window (can be increased)
🧠 Used SFT, PPO & DPO for alignment.
💰Commercial use allowed ✅
🤗 Available on @Hugging Face
🤝 1-click deployments on Hugging Face, Amazon SageMaker, Google Cloud
🔜 more model sizes & enhanced performance

Massive kudos to Meta for continuing its commitment to open AI. Honored to partner with Joe and team! 🤗 The gap is melting. 🧊

[clem 🤗 @ClementDelangue]

The new @MistralAI is now #1 on the openLLM leaderboard. Apache 2.0 license too! 🔥🔥🔥

[Philipp Schmid @_philschmid]

New Instances, New Region, New Capabilities! 🧠 @Google Cloud is now generally available on @huggingface! 🤗 We are excited to launch @GoogleCloudTech as an official backend for Inference Endpoints, offering you more options to power your Generative AI applications. 🚀

🌍 New Region, New instances with @nvidia L4 GPUs and H100 (on request)
🔒 Coming soon: Private Service Connect for secure VPC connection
🧠 Coming soon: Google TPUs with TGI support!
🤝 Coming soon: Marketplace integration for unified billing 📈

Help us prioritize! Which region and instance type are you most interested in? 🤔#GoogleCloudNext

[OpenAI @OpenAI]

Introducing Sora, our text-to-video model.

Sora can create videos of up to 60 seconds featuring highly detailed scenes, complex camera motion, and multiple characters with vibrant emotions.

https://t.co/YYpOAcrXQ3

Prompt: “Beautiful, snowy Tokyo city is bustling. The camera moves through the bustling city street, following several people enjoying the beautiful snowy weather and shopping at nearby stalls. Gorgeous sakura petals are flying through the wind along with snowflakes.”

[Philipp Schmid @_philschmid]

Code Llama 70B Instruct available in Hugging Chat! 💬

Try and experiment with @AIatMeta new Code Llama 70B for free in the @huggingface chat! 😍

👉 https://t.co/6HTfgra514

Share your experience in this thread! 🤗

[Santiago @svpino]

You can now access AI directly from your database!

Here is a step-by-step demo that uses GPT-4 to classify customer reviews from a MySQL dataset.

And I'm only writing SQL instructions!

You have to see it! The model acts as another table in the database. I can query it and join it with other tables. It's awesome!

This is all thanks to @MindsDB. Here is their GitHub repository: https://t.co/y5EsnseFTl. They are sponsoring this post.

Watch the video.

[Manjunath @_ssmanjunath]

@tunguz Somehow teach it back to gpt, so that we can get pretrained models + weights given prompt. InceptionGPT

[Santiago @svpino]

Another deep learning breakthrough:

Deep TDA, a new algorithm using self-supervised learning, overcomes the limitations of traditional dimensionality reduction algorithms.

t-SNE and UMAP have long been the favorites. Deep TDA might change that forever.

Here are the details:

[Manjunath @_ssmanjunath]

@kunalb11 How to minimise bias induced in AI training to get a task done ?

[Aran Komatsuzaki @arankomatsuzaki]

Training Diffusion Models with Reinforcement Learning

Presents an RL-based framework for training denoising diffusion models to directly optimize a variety of reward functions

https://t.co/XmQa7DGzDc