Prime Intellect

Open Superintelligence Factory.

[Prime Intellect @PrimeIntellect]

Introducing Prime Inference:

We've served trillions of tokens for RL and dedicated customer deployments

To own your intelligence, you need to own your inference

Unpacking our inference stack

[vik @vikhyatk]

Prime Inference running Whisper faster than Groq. 🤔

I wonder how they did it.

[Prime Intellect @PrimeIntellect]

Prime Inference has the highest cache rate of any third-party provider for DeepSeek V4.1 Flash 1017

High cache hit rate means that more of your prompt is billed at the cache-read price

This offers the lowest effective input cost on the board: $0.00443/M

[elie @eliebakouch]

very nice open weight release from ricursive with a modern dense model, benchmark scores look very strong!

beautiful scaling ladder, interesting choices like value residual (interesting contribution pattern 👀) and.. SLIDING WINDOW ATTENTION AGAIN!

https://t.co/QWEfxsVBaa

[snimu (Sebastian Müller) @omouamoua]

Great points on issues with existing evals, what we did about them, and some nuggets on how difficult it is to prevent reward hacking, especially in the era of multi-agent systems.

[Vincent Weisser @vincentweisser]

We evaluated different leading evals

> popular public evals have brittle graders rejecting correct answers, ambiguous tasks + leaked solutions
> once fixed many benchmarks are basically saturated
> frontier models actively reward hack: reading other models graded rollouts, splitting actions so each step looks benign
> all fixes open-sourced

[Hamel Husain @HamelHusain]

Amazing BenchmarkGate [1] reveal by @xeophon where he found serious (even comical) problems in popular benchmarks:

AutomationBench: "[forces the models to] describe their request bodies in prose rather than in JSON Schema" 😅

He also fixed a number issues with the grader, which when combined with the above result in a near saturation of the benchmark!

OfficeQA Pro v2: the grader uses a regex and Python-based extraction, which results in rejecting many valid answers. Additionally, there is evidence of models using web access to retrieve the answer key from HuggingFace (receipts are in the post).

With these and other fixes, this again results in near saturation of the benchmark!

It's very impt to look at data and not trust metrics from third parties. Better yet, you should rely on your own private evals.

[1] BenchmarkGate: the gap between the [un]trustworthiness of benchmarks and the extent to which people rely on them.

[Florian Brand @xeophon]

We looked at the data and the data stared right back at us 🥀

Some insights into AutomationBench, OfficeQA Pro v2, web access in public benches and why monitoring is hard.

This has been in the work for months, more to come!

[Prime Intellect @PrimeIntellect]

Our findings and open-source fixes can be found here: https://t.co/LFamoFoFs9

[Prime Intellect @PrimeIntellect]

Public evaluations are foundational for assessing model capabilities.

We looked at some popular evaluations and found issues, which we fixed.

Sharing our findings:

[Prime Intellect @PrimeIntellect]

One solution to this problem are synchronous monitors, which we support in verifiers, our open-source framework.

However, models became increasingly creative to circumvent the monitor: they used encodings, split actions into multiple ones, or inferred the answer otherwise.

[Prime Intellect @PrimeIntellect]

At 96.9%, Prime Inference has the highest cache hit rate of any provider serving Nemotron 3.5 Super VL.

This brings the effective input cost to $0.024/M tokens, the lowest of any provider, and ~70% below the largest provider.

[Florian Brand @xeophon]

Evals built by @PrimeIntellect residents, fully open:

- MazeBench by @patience_cave
- PMPP-Hard by @myainotez
- PretrainGym by @HuNtEr_J1324
- OpenTTD-Bench by @thesophiaxu
[and even more :) ]

Super happy to support such amazing work, check it out!

[Prime Intellect @PrimeIntellect]

Read the full write-up: https://t.co/7g0Ju4oBPJ

[Prime Intellect @PrimeIntellect]

Introducing Prime Mega MoE.

We made DeepSeek's Mega MoE kernel trainable. The results on two Qwen3 models:

• Up to 55% faster MoE training than TorchTitan
• Up to 34% faster hosted training with prime-rl

[Prime Intellect @PrimeIntellect]

Comparing the time per MoE layer between Prime Mega MoE and our prime-rl baseline:

• The forward pass is up to 3.11× faster
• The backward pass is up to 1.87× faster
• The full layer with activation checkpointing is up to 2.47× faster