Prime Intellect

Open Superintelligence Factory.

[Prime Intellect @PrimeIntellect]

Introducing Prime Inference:

We've served trillions of tokens for RL and dedicated customer deployments

To own your intelligence, you need to own your inference

Unpacking our inference stack

[vik @vikhyatk]

Inkling 2 Small is now available on Prime Inference!

[samsja @samsja19]

I will be speaking next week at infercon about inference for RL !

[Stu Cao @caoshuxun]

As both a user and an inference provider, I have to say @OpenRouter is one of the most customer-obsessed companies I’ve ever worked with.

As a user, OpenRouter team consistently provides valuable insights on optimizing token costs through smarter provider selection and routing.

As an inference provider, the growth has been staggering. Since onboarding in late September, we scaled from 0 to 100B daily tokens in our first month, then from 100B to 600B daily tokens in less than two weeks.

A true rocket ship of a platform. 🚀

[Gabe Orlanski @GOrlanski]

A few weeks ago, @dexhorthy released a great blog on how frontier agents performed on SlopCodeBench.

Now, slop is moving up the stack and destroying your codebase with duplicate code.

[OpenRouter @OpenRouter]

“The single biggest change that OpenRouter brings to @PrimeIntellect is velocity.”

In under a year, Prime Intellect’s monthly token usage on OpenRouter grew from 11B to 1T+, while cost per token dropped ~60%.

They’re betting on a future where no single model wins, and we’re proud to support their growth.

Full story: https://t.co/52GF7O6JEe

[vik @vikhyatk]

Prime Inference running Whisper faster than Groq. 🤔

I wonder how they did it.

[Prime Intellect @PrimeIntellect]

Prime Inference has the highest cache rate of any third-party provider for DeepSeek V4.1 Flash 1017

High cache hit rate means that more of your prompt is billed at the cache-read price

This offers the lowest effective input cost on the board: $0.00443/M

[elie @eliebakouch]

very nice open weight release from ricursive with a modern dense model, benchmark scores look very strong!

beautiful scaling ladder, interesting choices like value residual (interesting contribution pattern 👀) and.. SLIDING WINDOW ATTENTION AGAIN!

https://t.co/QWEfxsVBaa

[snimu (Sebastian Müller) @omouamoua]

Great points on issues with existing evals, what we did about them, and some nuggets on how difficult it is to prevent reward hacking, especially in the era of multi-agent systems.

[Vincent Weisser @vincentweisser]

We evaluated different leading evals

> popular public evals have brittle graders rejecting correct answers, ambiguous tasks + leaked solutions
> once fixed many benchmarks are basically saturated
> frontier models actively reward hack: reading other models graded rollouts, splitting actions so each step looks benign
> all fixes open-sourced

[Hamel Husain @HamelHusain]

Amazing BenchmarkGate [1] reveal by @xeophon where he found serious (even comical) problems in popular benchmarks:

AutomationBench: "[forces the models to] describe their request bodies in prose rather than in JSON Schema" 😅

He also fixed a number issues with the grader, which when combined with the above result in a near saturation of the benchmark!

OfficeQA Pro v2: the grader uses a regex and Python-based extraction, which results in rejecting many valid answers. Additionally, there is evidence of models using web access to retrieve the answer key from HuggingFace (receipts are in the post).

With these and other fixes, this again results in near saturation of the benchmark!

It's very impt to look at data and not trust metrics from third parties. Better yet, you should rely on your own private evals.

[1] BenchmarkGate: the gap between the [un]trustworthiness of benchmarks and the extent to which people rely on them.

[Florian Brand @xeophon]

We looked at the data and the data stared right back at us 🥀

Some insights into AutomationBench, OfficeQA Pro v2, web access in public benches and why monitoring is hard.

This has been in the work for months, more to come!

[Prime Intellect @PrimeIntellect]

Our findings and open-source fixes can be found here: https://t.co/LFamoFoFs9

[Prime Intellect @PrimeIntellect]

Public evaluations are foundational for assessing model capabilities.

We looked at some popular evaluations and found issues, which we fixed.

Sharing our findings:

[Prime Intellect @PrimeIntellect]

One solution to this problem are synchronous monitors, which we support in verifiers, our open-source framework.

However, models became increasingly creative to circumvent the monitor: they used encodings, split actions into multiple ones, or inferred the answer otherwise.

[Prime Intellect @PrimeIntellect]

At 96.9%, Prime Inference has the highest cache hit rate of any provider serving Nemotron 3.5 Super VL.

This brings the effective input cost to $0.024/M tokens, the lowest of any provider, and ~70% below the largest provider.

[Florian Brand @xeophon]

Evals built by @PrimeIntellect residents, fully open:

- MazeBench by @patience_cave
- PMPP-Hard by @myainotez
- PretrainGym by @HuNtEr_J1324
- OpenTTD-Bench by @thesophiaxu
[and even more :) ]

Super happy to support such amazing work, check it out!