Institute of Foundation Models

We create world-class foundation models that redefine what AI can do.

[Institute of Foundation Models @IFM_AI]

Introducing K2 Horizon: a connected fleet of six foundation models ranging from 0.9 billion to 375 billion parameters.

- Frontier performance: Across coding and agentic tasks, K2 Horizon delivers top-tier performance in every size class—with the 0.9B, 3.7B and 7B models setting new state of the art at their respective scales.
- Radical openness: K2 Horizon represents the largest fully open-source model launch in AI history. The fully open code, training data and recipes are a significant step forward in transparency.

Launch page: https://t.co/gg0k803SbL
Tech blog: https://t.co/g35L5xMGdS
Hugging Face: https://t.co/3Lb28JhyG9

[Institute of Foundation Models @IFM_AI]

A teacher model is limited by its own abilities. But the training data we generate doesn't have to be.

Two examples:

- Long coding sessions: Replay repository PR-based tasks. The teacher handles one task at a time, but the training trajectory captures the entire session, reaching hundreds of thousands of tokens.
- Privileged advisor: Use hidden-test feedback to identify mistakes and directly edit a solver's reasoning with targeted cues.

[Institute of Foundation Models @IFM_AI]

The most detailed open LLM data release yet: K2 Horizon's pre-training data and post-training data-generation pipelines are now open!

13 trillion pre-training tokens across 5 datasets and 24 subsets. Over 10 trillion tokens of synthetic data spanning mathematics, reasoning, coding, planning, and problem-solving.

Beyond the scale, four highlights for model builders (details in the threads):

- Scaling synthetic data without losing diversity. Our generation methods produce diverse training examples at scale, nearly matching natural web text in measured diversity.
- More verification isn't always better. Strict verification can throw away useful training signal.
- Small-scale wins can mislead. Our chosen data mix actually lost in smaller experiments.
- Beyond teacher imitation. Our post-training data generation methods produce training trajectories the teacher couldn't generate on its own.

Open pre-training datasets, post-training data-generation pipelines, and processing tools for researchers to inspect, reuse, and build on.

Pre-training data: https://t.co/3Lb28JhyG9

Post-training data-generation code & tools: https://t.co/YWgJpOnzVe

[Institute of Foundation Models @IFM_AI]

The best data mix in a small experiment isn't necessarily the best at scale.

Our web-heavier mix ranked last in smaller experiments. But our scaling predictions put it first for larger models trained on 20T tokens. So that's the mix we chose.

The lesson: optimizing data mixtures for quick proxy experiments may not give the best results at the scale you're targeting.

[Institute of Foundation Models @IFM_AI]

What’s next: every lesson points the same way. Isolate failures, distribute everything end to end, schedule dynamically, move less data, and push past 10K samples per step.

We’re building that into Wrench, our next framework. Coming soon.

[Institute of Foundation Models @IFM_AI]

We're open-sourcing RL360, the RL infrastructure behind K2 Horizon's six domain experts: 8,192 concurrent sandboxes, 150 steps in 7 days, ~1.3M rollouts.

Long-horizon agentic RL runs thousands of agents in real sandboxes on tasks that take minutes. Some of those sandboxes crash mid-task, and a crash arrives as the same low score as a wrong answer.

Without infrastructure that can keep up, GPUs are underutilized and the model gets penalized for system crashes rather than its own mistakes.

RL360 is built for this. We share how it works, what broke at scale, and what we learned in the thread.

Code: https://t.co/eA46QBpfTx
Run: https://t.co/cd4tOV3OXv

[Institute of Foundation Models @IFM_AI]

Lesson 3: run images where they’re already cached. Sandboxes that landed on random nodes kept pulling their images cold. Images are pinned to specific nodes, and most sandbox starts hit a warm local cache.

[Institute of Foundation Models @IFM_AI]

Check out the Leaderboard: https://t.co/2DRk56PyOH

[Institute of Foundation Models @IFM_AI]

Jais-ASR ranks #1 on the Open Universal Arabic ASR Leaderboard.

Our open model is built for Modern Standard Arabic and dialects from 19 countries.

Beyond the leaderboard, it handles:
- Natural code-switching with English, French, Spanish, and Italian
- Robustness to background noise
- Faithful transcription, resistant to hallucinations
- Arabic diacritization
- Built-in language and dialect identification

Weights released under Apache 2.0: https://t.co/kXjQEqVXvU

[Institute of Foundation Models @IFM_AI]

More technical details: https://t.co/BGHWASCr9W

[Institute of Foundation Models @IFM_AI]

Get your API keys: https://t.co/W6IrYtCrwb

[Institute of Foundation Models @IFM_AI]

Jais 3 is built for Arabic from the start.

Speech: the most accurate open Arabic ASR, across MSA and dialects

Writing: fluent, native Arabic from the first line to the last

Agents: leads open models on Arabic agentic tasks

Download weights: https://t.co/Q2ZI1PJxAW

[Institute of Foundation Models @IFM_AI]

Day-0 support for Jais 3 on @sgl_project, so an Arabic-first agentic model is now straightforward to serve at scale. Thank you to the SGLang team!

Weights: https://t.co/Q2ZI1PJxAW
API: https://t.co/r6vG73baqU

[SGLang @sgl_project]

Jais 3 from @IFM_AI is live on SGLang from Day 0!

Jais 3 is a family of six open Arabic-centric models, from 0.9B to 375B, built to plan, call tools, and work on Arabic documents as agents, ready to serve on SGLang!