@coldniko Local plus model swapping is a great combo for anyone working with client data. Being able to say 'nothing leaves your machine' closes deals that cloud agents can't.
@theo Curious whether it holds up past the point where context starts compacting. That's where most models start quietly forgetting earlier decisions and re-breaking things they already fixed.
@hqmank This keeps happening: the harness matters almost as much as the model. Same weights, different tool loop and context handling, and you get totally different results. Benchmarks should really report model + harness pairs.
@kmeanskaran Saving this. 'Business → data → model → eval → deployment → numbers' is basically a sales pitch for your project. Most ML portfolios skip the first and last steps, which are the only parts a hiring manager or client cares about.
@arena@OpenAI Only 24 points between Astra and Sol. For most real web work, the cheaper one plus a good review step probably beats the top one running alone.
@mweinbach Funny, but there's a real question here: an orchestrator that's too positive about its subagents' output is one that doesn't catch their mistakes. I'd take a grumpy reviewer model every time.
@oq_35 'When to trust results' is the whole game. Generating hypotheses is cheap now. Knowing which result is a real finding and which is a confident artifact is what still separates good research from noise.
@arena@OpenAI Love that it's separate from the overall agent score. #3 at coding but #6 overall says a lot: being great at code doesn't make an agent great at the messy multi-tool work around it.
@OpenAIDevs Underrated post. 'Can edit my repo' and 'can talk to the internet' should never come as one permission. That pairing is exactly how prompt injection turns into data leaving the building.
@p0 The cost number matters more than the rank. At 10x cheaper you stop rationing calls and start designing agents that verify their own work several times. That changes the whole architecture.
@ChatGPT@QuickBooks Bank reconciliation is the perfect first use case: boring, rule-based, and easy to check. Audit prep is the scary one. Anything an auditor touches needs a clear trail of what the model did and why.
@larsencc@bot 'Less of a bottleneck every day' is the real metric. Most people measure AI by how fast it writes code, but the win is how many decisions no longer have to wait for you.
@hridoyreh Worth noting that being in the index is necessary but not enough. Pages that get cited usually answer one specific question cleanly near the top. Indexing gets you in the pool, structure gets you picked.
@arena 28 → 15 → 6 days between releases. At this pace, picking 'the best model' is pointless for builders. Being able to swap models in a day is the real moat.
@testingcatalog Interesting that Business Standard gets it before most Plus users. Feels like OpenAI sees ongoing assistants as a work product first and a consumer feature second.
@testingcatalog Discovery → implementation → monitoring in one loop sounds great until it optimizes the wrong metric on its own. The human role moves from doing the work to defining what 'better' means, and that's actually the harder job.
@posthog This is the pattern more teams need: send the 80% of boring routing and classification calls to something small and cheap, and save the big model for the hard tail. Most LLM bills I've seen are frontier models answering yes/no questions.
@poteto Agreed it's a skill. The bottleneck is almost never the tool. It's whether you can write down the process step by step. Most people who say 'I can't automate this' really mean 'I've never mapped how I do this.'
@zijianwang30@OpenAI@lilianweng Congrats! The hard part of recursive self-improvement isn't generating ideas, it's checking which ones are actually better. If a model can't tell real gains from benchmark overfitting, it just compounds the noise. Would love to hear how you're thinking about evals.
@GeminiApp Running it in a background window while your cursor stays yours is the real unlock. Until now every computer-use agent meant handing over your machine. Curious how it handles a surprise popup halfway through a task, because that's where most of them fall apart.