Mohit Parmar

[Ashish Vaswani @ashVaswani]

Rnj-1-Instruct is now the #1 trending text generation model on HF!

[Yash Jain @_jainyash]

rnj-1 is now #1 trending on @huggingface. Go test it out!

[Essential AI @essential_ai]

We are now the #1 trending text-gen <256B size model on HuggingFace!!

[Essential AI @essential_ai]

Proud to carry the spirit of Ramanujan with rnj-1: honoring the great minds who showed how curiosity reshapes the world. The lineage of deep thinking is part of our mission. Essential proud.

[Essential AI @essential_ai]

Check it out https://t.co/KJZjBJHzFB

[Essential AI @essential_ai]

Rnj-1 is now live on @openrouter!

[Ashish Vaswani @ashVaswani]

We are beyond thrilled to share our first flagship models, Rnj-1 base and instruct 8B parameter models. Rnj-1 is the culmination of 10 months of hard work by a phenomenal team, dedicated to advancing American SOTA OSS AI.

Lots of wins with Rnj-1.

1. SWE bench performance close to GPT 4o.
2. Tool use outperforming all comparable open source models.
3. Mathematical reasoning (AIME’25) nearly at par with GPT OSS MoE 20B.

….

[Ashish Vaswani @ashVaswani]

@timr1126 @ashehper @ameya_pa @gordic_aleksa @kurtosis0 @mobydobius @Khoi_H_Nguyen @Mperor11 made fundamental improvements in data curricula and synthetic data transformations to to teach our models advanced Math reasoning skills.

[Essential AI @essential_ai]

Today, we’re excited to introduce Rnj-1, @essential_ai's first open model; a world-class 8B base + instruct pair, built with scientific rigor, intentional design, and a belief that the advancement and equitable distribution of AI depend on building in the open.

We bring American open-source at par with the best in the world.

[Ritvik Kapila @ritvikkapila]

#1 trending on @huggingface letsgoooo!

@essential_ai 🥇

[elie @eliebakouch]

24T token dataset but the best part is that it's labelled with a 12-category taxonomy covering subject, reasoning depth, education level and other quality signals

[Essential AI @essential_ai]

[1/5]

🚀 Meet Essential-Web v1.0, a 24-trillion-token pre-training dataset with rich metadata built to effortlessly curate high-performing datasets across domains and use cases!

[Rohan Paul @rohanpaul_ai]

A 24-trillion-token web dataset with document-level metadata just dropped on @huggingface

License: apache-2.0

ESSENTIAL-WEB v1.0 collects 24 trillion tokens from Common Crawl. Each document is labeled with a 12-field taxonomy covering topic, page type, complexity, and quality .

Labels are generated by EAI-Distill-0.5b, a 0.5B-parameter model fine-tuned on Qwen2.5-32B-Instruct outputs. It matches teacher quality within 3% agreement and preserves domain recall within 1pp .

Simple SQL-style filters produce datasets competitive with specialist pipelines. Math is within 8% of SOTA, web code up 14.3%, STEM up 24.5%, and medical up 8.6% .

Inference on 23.6 B documents required about 90k AMD MI300x GPU-hours.

[Ashish Vaswani @ashVaswani]

Check out our latest research on data. We're releasing 24T tokens of richly labelled web data. We found it very useful for our internal data curation efforts.

Excited to see what you build using Essential-Web v1.0!

[Mohit Parmar @Mperor11]

RT @essential_ai: Muon is a serious competitor to AdamW, but it's tricky to scale up.

Our infra team has made fundamental advancements in…

[clem 🤗 @ClementDelangue]

Cool paper from @essential_ai @ashVaswani https://t.co/lypRuhtOhp

[Ashish Vaswani @ashVaswani]

Please check out our thorough study on the advantages of Muon. Second-order optimization is a promising path to more efficient LLM pretraining.

[AI at AMD @AIatAMD]

Pretraining just got smarter.

Even a 7B model trained on 198B tokens can reflect—reasoning during pretraining, not just fine-tuning.

@KarimBhalwani shares how: https://t.co/NPd0pi9ptg

[Karim @KarimBhalwani]

I’m excited to share my first blog post for @AMD: “Pretraining Just Got Smarter”

In it, I highlight @essential_ai's latest research showing that reasoning can emerge during pretraining- not just through post-training or RL.

Powered by AMD Instinct™ MI300X GPUs, Essential AI ran these experiments on a Kubernetes-managed cluster, serving models through vLLM and SGLang.

It’s a great example of the kinds of breakthroughs made possible when open science meets scalable compute.

Thank you @ashVaswani, @kurtosis0, @ssingla17, @Mperor11, Darsh Shah, Pete Rushton & Essential AI Team for redefining the starting point for reasoning in language models.

Thank you @AIatAMD for enabling us!

@AMD 💕 @essential_ai

🔗 Read the blog https://t.co/rgFMzPBsj8

📄 Research paper
https://t.co/qpoAS2Cs75

#AI #LLM #OpenScience #MI300X #AMD #EssentialAI

[elvis @omarsar0]

Rethinking Reflection in Pre-Training

This is one of the more interesting papers I read this week.

It argues that reflection emerges during pre-training.

It introduces adversarial reasoning tasks to show that self-reflection and correction capabilities steadily improve as compute increases, even in the absence of supervised post-training.

Key contributions:

• Propose two kinds of reflection:
- Situational-reflection: The model corrects errors in others’ reasoning.
- Self-reflection: The model corrects its own prior reasoning.

• Build six adversarial datasets (GSM8K, TriviaQA, CruxEval, BBH) to test reflection across math, coding, logic, and knowledge domains. On GSM8K-Platinum, explicit reflection rates grow from ~10% to >60% with increasing pre-training tokens.

• Demonstrate that simple triggers like “Wait,” reliably induce reflection.
• Evaluate 40 OLMo-2 and Qwen2.5 checkpoints, finding a strong correlation between pre-training compute and both accuracy and reflection rate.

Why it matters:
• Reflection is a precursor to reasoning and can develop before RLHF or test-time decoding strategies.
• Implication: We can instill advanced reasoning traits with better pre-training data and scale, rather than relying entirely on post-training tricks.
• They also show a trade-off: more training compute reduces the need for expensive test-time compute like long CoT traces.