Roger Weiss

Geek - my opinions are my own and do not necessarily reflect that of my employer.

[Roger Weiss @Roger_D_Weiss]

@YahooFinance “We can’t wait to see what you’re going to build on our silicon” 🙄

[Roger Weiss @Roger_D_Weiss]

@morganlinton @VulcanBench Very interesting and surprising. Compared to your previous Sol benchmark with Opus and Sonnet, it feels like Haiku is on the same ballpark as Sonnet at Max?

[Roger Weiss @Roger_D_Weiss]

@SJP1804 Yes! Rust code is more readable than the Bible, and verifiable!

[Roger Weiss @Roger_D_Weiss]

@BBCWorld Rich, coming from the days when Internet Explorer 6 was unregulated.

[Roger Weiss @Roger_D_Weiss]

@BBCWorld Yes but the one is accelerating intelligence.

[Roger Weiss @Roger_D_Weiss]

@py_poojaa Omarchy.

[Roger Weiss @Roger_D_Weiss]

@johncodes How many contributors do you want on LibreOffice.

Also my money, my tokens, no need for socialist tendencies on how people should use their tokens .

[Roger Weiss @Roger_D_Weiss]

@mayuri_3015 Zed

[Roger Weiss @Roger_D_Weiss]

@AusMachiavelli @TheCleanCarClub It’s time to switch to Chinese Models. All of them look so good!

[Roger Weiss @Roger_D_Weiss]

@verge So that Microsoft can catch up?

[Roger Weiss @Roger_D_Weiss]

@shillhousecom Good, but Just don’t try use two pedals at once . Play and Pause together sounds like fun, but not a good idea

[Roger Weiss @Roger_D_Weiss]

@ApplesClubs Pity that one comes with Windows.

[Roger Weiss @Roger_D_Weiss]

@vovudebosh I wish this ad would go away.

[Roger Weiss @Roger_D_Weiss]

@mayuri_3015 9..

[Roger Weiss @Roger_D_Weiss]

@theo Concise presentations about coding.

[Roger Weiss @Roger_D_Weiss]

@Ahmed69996059 Wrecked

[Roger Weiss @Roger_D_Weiss]

@theapplecycle Pity the Surface comes with Windows. Great specs otherwise. In the meantime I’ll stick with MacBook, any MacBook, even the Air.

[Morgan @morganlinton]

Okay, just got the results of my Sonnet 5.5 benchmark with @VulcanBench on my Frontier v4 eval suite.

Super interesting, not what I was expecting.

It's amazing how many times my own benchmarks surprise me.

While both Opus 5.5 and Sonnet 5.5 do score higher than GPT-6.1 Sol, it's not by much, and Sol is just sooooo much more cost effective.

At the same time, looking at Opus 5.5 specifically, it is definitely ahead enough, especially and Medium and High effort levels, that I do see a strong reason to give it your hardest tasks.

But for easy and medium difficulty stuff, not really sure why you wouldn't use GPT-6.1 Sol, the cost-per-task is just insanely low.

I test all Claude models in Claude Code and OpenAI models in Codex, so could also be that @thsottiaux and team are working some magic with Codex to help with token efficiency, because this is pretty wild.

Adding to the VulcanBench site and Github soon so everyone can dig into the data.

Everything I do is independent and open source, you can look at the data from every single run, even run it all yourself if you want to.

Live long and benchmark 🖖

[Roger Weiss @Roger_D_Weiss]

@HackingBaseball Yup, don’t need validation. Python guys be like “I use Python BTW” 🤣