@DanKornas Passive capture from session turns is the interesting bit. But a lot of the 'no' happens in PR review, not in the chat with the agent. Does RoBrain pick up rejections from review comments too? That's where I'd expect a rejected approach to come back.
@theo In that chart Argon had 47 of 92 checked claims marked false, vs 2 to 6 for the others. It also ran at 'high' while the others ran at 'max', so some of the gap may be effort. Were the false claims invented names, or wrong readings of real code?
@gregisenberg 'Tell it to make the tests pass and it might rewrite the tests.' Same fix as Airbnb pairing bookings with cancellations: pair 'tests green' with 'test files untouched'. If the agent edits a test, that diff goes to a human before merge, not in with the rest.
@marketcallsHQ "Testing builds with live brokers" is the step I'd never automate away on a trading platform. Level 5 drops code review, fine. But who signs off on what touches real orders? I'd argue that gate is part of the product, not a level you graduate from.
@undefinedKi Fair on the reordering, same tokens in a different order. The part I'd test is 'cut tools from 16 to 4'. That changes what the reviewer can look up beyond the diff. If a dropped tool was how it read callers or tests, that's where a planted bug slips through.
@Hesamation Zero privileges, with more access given only if necessary, is the right default. The hard part is who decides 'necessary' mid-task. If a human clicks yes all day, approval turns into a reflex. Is that call made by policy or by a person in OpenShell?
@Hesamation "Compete with everyone else's frontier model without even using its own frontier" would mean public leaderboards rank the students, not the teachers. If that's right, today's benchmarks are a lagging signal. The real question is how fast the next student ships.
@Voxyz_ai "Don't treat a task as finished just because another agent says it is" is the rule that matters most here. The bug scoring is smart too. But with 10 workers, the final review is where the queue builds up. Who checks what Opus integrates?
@vasuman "SDLC has seen 99% of the benefits" is where I'd push back. Writing code got faster. Review, testing and deciding what reaches production still need a human, and that's where the queue builds up now. More code means more to check, not less.
@mardehaym "A measure based on what shipped" is the line that makes the rest work. Asking people how they use AI grades the habit, not the output. What do you count as shipped though: merged, or still in production a month later?
@kimmonismus "5 weeks after its release" and still on top is the real story. If Fable 5.5 and a new Opus land the same week, "best overall" stops being the question. Teams with a harness tuned to Opus 5.5's habits will ask what breaks if they switch. Swap on day one or wait for your evals?
@DanielSmidstrup Neither first. The role squeezed first sits in between: turning a clear ticket into code, which is exactly what agents are good at. A junior who can read a diff and say no, and a CTO who decides what's worth building, both get more valuable. Typing was never the job.
@undefinedKi Point 3 decides everything. "Without it, you are the harness" still holds with one: if the same model writes the code and the check before "done", you're still the last check. Who writes the tests matters more than the rules file.
@jackfriks "As long as it works and is secure" is doing the heavy lifting there. The rewrite in 2 hours is the easy half now. Knowing the Kotlin version handles auth, storage and permissions the way the Swift one did is the half that doesn't fit in 2 hours. How did you check it?
@kimmonismus On the RSI point I'd want receipts. "Focusing on RSI" can mean the model writes its own training data, grades its own evals, or picks the merge. Very different claims. Which one are the Chinese labs actually doing?
@wickedguro "No more unupdated packages" and "partners that might get hacked" pull in opposite directions. Auto-updating everything is how a hijacked package can reach you. Pin versions, update with a delay, read what changed. Fresh isn't the same as safe.
@mardehaym "Then you're just engineers" is the line every agency should frame. Once agents write most of the code, hours stop being the product. What the buyer can't hire is someone who kills the wrong project before a single line gets written.
@DanielSmidstrup Outside X, I'd guess most people don't buy an AI tool. Their company does, bundled inside something they already open every day: the IDE, the office suite, the CRM. Three subscriptions is the hobbyist market. The bundle is the real one.