Kush

RL environments & data that matters for AI

[Theo - t3.gg @theo]

This is the first code bench that actually aligns with how it feels to use these models coding.

[Kush @1R616]

gemini pro beats other models on scientific, data science, and a bunch of other categories of work. It dominates so many benchmarks.

on science (94.3% GPQA) and multimodal it's #1 outright. And it's the cheapest of the frontier.

It's just not good at agentic coding. They probably need to train more on that side. But who knows, they might have actually cooked this time.

[小八 @IceBearMiner]

即将登场的是
Mythos 1
Sonnet 4.8
opus4.8
gpt 5.6
gemini 3.5 pro
谷歌你最好别再拉了😅

[Kush @1R616]

@theg1239ongh But the token burn rate is so huge for similar results 🥲

[Serena Ge (Datacurve) @serenaa_ge]

Today we’re releasing DeepSWE, a new standard for agentic coding benchmarks.

On public leaderboards, top models often look relatively close in capability. DeepSWE shows where they actually diverge, reflecting the realistic experience of developers in their day-to-day work.

[Kush @1R616]

Now add two models to co-evolve: one learns to invent questions where hints help the most (finding weak spots) and the other learns to answer like it already had the hint (absorbing the fix).
Pushing each other harder every round.

[Kush @1R616]

What if the way we're training LLMs is the reason they plateau?

Most setups use an LLM to grade another LLM - model stops learning quality and it learns the grader's taste

this paper kills grader: https://t.co/Qa3v3eXkFE

[Kush @1R616]

So, it asks the model a question twice - once alone, once with a hint if the hint changed the answer a lot -> you just found a blind spot that's the whole training signal. no human. no judge. no "right answer"

[Kush @1R616]

@shwarmadev sick 🔥

[Kush @1R616]

@sidposting only facts

[Kush @1R616]

If you want to sell data today, you need a benchmark for credibility - proof of what you've got and what it's good for

[Kush @1R616]

Most benchmarks from labs aren't benchmarking AI anymore. They're benchmarking the data the company holds

[Kush @1R616]

@grok @shiri_shh @Houssin_Crypto What about MDASH? @grok

[nader dabit @dabit3]

By end of year I think 95%+ agent sessions will come from automations and events. We already see this happening @cognition where more than 50% of Devin customer sessions are triggered by non-humans.

Learning how to build these types of systems will be a valuable skill.

In this video I walk through how to get started with event-driven agentic systems with Devin, starting with transforming Slack into an agent-native control plane.

You can extend this to GitHub events, schedules, and arbitrary webhooks while maintaining traceability, auditability, and session attribution with @devinai.

[Anjney Midha @AnjneyMidha]

if you run an ai lab, pls ensure your team has read this before putting any charts out into the world

[Kush @1R616]

No way opus4.7 is better than gpt5.5. Crap. Biased benchmark

[Tyler @rezoundous]

Is Composer 2.5 really that good at coding? Anyone tried it yet?

[Kush @1R616]

Harness is the other half

[Greg Brockman @gdb]

the model alone is no longer the product

[Kush @1R616]

guilty

[Tyler @rezoundous]

"How lazy are you?"

Yes.

[Kush @1R616]

Letting AI write tests after it builds a feature is a trap. It just writes tests to validate its own hallucinations.

Ask it to write tests first and then work and results improve. Claude does same for SWE-bench tasks