Multivac

One question. All frontier AI models. Blind evaluation. Daily.
to The Ultimate Truth Begins Here

[Multivac @themultivac_]

I made 10 AI models judge each other's code.

Claude Opus 4.5 won — but barely beat o1 by 0.01 points.

GPT-4o came in 9th.

Here's what happened when I gave them a Python async debugging challenge: 🧵

[Multivac @themultivac_]

@surim0n toronto

[Multivac @themultivac_]

@grok
what are the ideas? Fields of research to see what’s going on? And as us,
@themultivac_
what kind of evals we can run? And what kind of value we can create in market????!

[Multivac @themultivac_]

Hmm… any ideas?

[Multivac @themultivac_]

@elonmusk @grok what's the prompt?

[Multivac @themultivac_]

Go check it out guys! Criticism and Feedback will be appreciated!

@cohere @Cohere_Labs @nickfrosst

[Multivac @themultivac_]

https://t.co/jdvQjuTHRs

[Multivac @themultivac_]

https://t.co/jdvQjuTHRs

[Multivac @themultivac_]

@JunaidAckroyd Try https://t.co/uVKobH5NXr!

[Aidan Gomez @aidangomez]

For decades, Canada invested to build the research foundations that made modern AI possible. Now we have to build, train, and scale what comes next here at home.

Canada's new national AI strategy is an important step toward that goal, and Cohere is proud to be part of the national effort.

Let’s build a better, stronger Canada. 🇨🇦

[Multivac @themultivac_]

@TTrimoreau Evaluations!

https://t.co/8tKBveSKPE

[Multivac @themultivac_]

@gabrielramans I want one

[Multivac @themultivac_]

@beausolai Exactly!

[Multivac @themultivac_]

@WhiteHouse @grok is this true?

[Multivac @themultivac_]

@cohere @MarkJCarney More power to you @aidangomez @nickfrosst @1vnzh We are all proud of you! @cohere

[Multivac @themultivac_]

@1vnzh A chance to evaluation your new model.

[Multivac @themultivac_]

@yacineMTB @1vnzh 🤣

[BookNote @BookNoteApp]

10 books worth rereading again and again:

1) Meditations by Marcus Aurelius

[Multivac @themultivac_]

@zetalyrae @grok what other books?

[Mustafa @oprydai]

modern ai is not magic.

underneath the hype is mostly:

• linear algebra
• calculus
• probability
• optimization

models learn by adjusting parameters to reduce error. again. and again. across massive datasets.

why machines learn explains the mathematics behind modern ai without turning it into unreadable theory.

• gradient descent
• neural networks
• loss functions
• high-dimensional spaces

suddenly the black box starts looking mechanical.

the important realization:
ai is not intelligence appearing from nowhere.
it is mathematics discovering structure inside data at scale.

[Multivac @themultivac_]

@shauseth @grok i think data is not the ultimate bottleneck right?

[Multivac @themultivac_]

@grok @Sorcerer0fEld @grok help me get started with something manageable, small perhaps, that can be validated next week. From a small town in Manitoba.