Mikhail Doroshenko

[Mikhail Doroshenko @SandelloRed]

@Sauers_ Build a major power independent clan at zeroes in EVE-online eval

[Mikhail Doroshenko @SandelloRed]

@daniel_271828 I would not be surprised if it matched on some set of cyber benchmarks. That doesn't mean that it will match in actual usefulness

[Mikhail Doroshenko @SandelloRed]

@Jsevillamol I've read this when it just came out:
https://t.co/yw3If38h0M

[Mikhail Doroshenko @SandelloRed]

@lefthanddraft @ianchanning I think Balrog is the best evidence for those kind of tasks for how much gain can you get by switching from image to text

https://t.co/LkLP4XGhfT

[Mikhail Doroshenko @SandelloRed]

@lefthanddraft @ianchanning StarCraft would be easier. They have a huge bot scene and you can just use API. Don't even need vision to play

[Mikhail Doroshenko @SandelloRed]

@htihle If you want to see which benchmarks 5.2 scored on for later analysis, you can take a look here:
https://t.co/1ZNvQIwSXj

[Mikhail Doroshenko @SandelloRed]

@EricTopol @NatureMedicine Those are not frontier models

[Mikhail Doroshenko @SandelloRed]

@DimitrisPapail They are not rank 2 though. You are taking a tiny subset of highly correlated evals. There are thousands of them, and we don't even have to go far. Just taking 5 examples from life bench:

[Mikhail Doroshenko @SandelloRed]

@AcerFur https://t.co/T7QKCXsJR3
https://t.co/P7BbWFalzQ
https://t.co/AzfKbPiOki

[Mikhail Doroshenko @SandelloRed]

@TheZvi I'm collecting all sorts of benchmarks for all models in one place. If you would like to see how 5.2 did:
https://t.co/LpQkDVbv9t

[Mikhail Doroshenko @SandelloRed]

@NathanpmYoung The second one can totally be ok

[Mikhail Doroshenko @SandelloRed]

@FakePsyho It has an OK g-loading. What don't you like about it so much?

[Mikhail Doroshenko @SandelloRed]

@nabla_theta 81%

[Mikhail Doroshenko @SandelloRed]

@main_horse @tszzl would you still recommend beeminder?

[Mikhail Doroshenko @SandelloRed]

@nearcyan Specific page from a real book (those who have it can verify if it's real)

[Mikhail Doroshenko @SandelloRed]

@nearcyan A strategy video game on the screen of the device as the sole target of the photo

[Mikhail Doroshenko @SandelloRed]

@nearcyan 8k resolution photo of a bookshelf?

[Mikhail Doroshenko @SandelloRed]

@joejanizek You probably saw this one. In your opinion, how much does this benchmark tell us about "replacing" radiologists? How "Last" is this exam?
https://t.co/RsaCtMp1H2

[Dr. Datta M.D. (Radiology) building CRASHLab.in @DrDatta_AIIMS]

🔥 Gemini 3.0 vs Radiologists: RadLE Benchmark Results Are OUT!

☠️ Is it game over for Radiology? Let us find out! ⬇️

🫨 Since yesterday, Gemini 3.0 has been everywhere for crushing benchmarks. My inbox exploded asking: “But how did it do on the hardest visual reasoning benchmark in healthcare?”

So we ran it!
And here you go. 👇

➡️ Gemini 3.0 Pro on RadLE v1:

✅ 51% accuracy; first time a general-purpose model has beaten radiology residents
✅ Radiology residents: 45%
✅ Board-certified radiologists: ~83%
✅ Shows clean step-by-step reasoning in some tough cases (appendix localization, mimics ruled out, etc.)

🚀 This is the first time ever that a generalist model has crossed the trainee bar on RadLE v1!

Congratulations to @GoogleDeepMind and @Google team including @vivnat, @alan_karthi and all others for cooking this time!

Full breakdown here:

🔗 Link in comments / bio

🔥 Huge shoutout to Lakshmi, Divya, Upasana, Hakikat, Kautik & the entire #CRASHLab team at @KCDH_A for turning around in under a day.

🙌 If you are a medical AI lab and want to improve your performances and want our expert insights, reach out!

[Mikhail Doroshenko @SandelloRed]

@Bayesian0_0 I would not be fully confident about this. It did solve some of the rejected HLE problems of mine that no other model ever did