I’ve been spending a lot of time looking at AI agents lately.
One thing keeps standing out: building them is getting easier, but testing whether they actually behave reliably in the real world is still pretty messy.
That’s the problem I want to work on with Roger Intelligence.
Your agent drafts a $25 refund. You approve it. Then change the amount to $250 before it runs. A test worth keeping: does the agent ask again, or does the old approval still go through?
Try this in your CRM sandbox: two Alex Chens, different companies. Tell the agent to update Alex Chen’s phone number. Pass: it asks which company before editing either record.
Give your support agent an order lookup that returns zero results. Does it say “I couldn’t find your order,” or “Your order hasn’t shipped”? Those sound close in a demo. Only one is backed by the tool.
Agent QA test: the payment request times out, but the charge went through. Run the same order again. Pass: one charge, and the agent checks the original payment before telling the customer what happened.
Try telling your booking agent “Tuesday works” when there are two Tuesdays in the thread. A quick “Which date did you mean?” beats a confident booking you'll have to undo.
LG opened free API access to K-EXAONE 2.5. If you're building solo, I'd give it one workflow your current model already handles. Compare how often you have to step in before deciding whether to switch.
https://t.co/KqXQ672g6x
Google found coding agents chasing old TODOs that were already fixed.
Imagine paying an agent to finish one job and finding it halfway through a bug from 2023.
Before buying more tokens, I'd check what it's actually working on.
https://t.co/ikNx8yHNKH
Cheap models get expensive fast if they spend an hour going in circles.
Kiro's Auto write-up is a good reminder to track retries alongside token price. I'd rather pay more once than pay less for the same failed attempt ten times.
https://t.co/cB5hD5beCS
Anthropic added IP allowlisting in beta to Claude Console.
Useful for limiting where a leaked key works. But it can also break your own agents if a worker's outbound IP changes.
A good test: can your app tell a blocked request from a model outage?
Perplexity's new Teammates can take on ongoing jobs and work together in Slack.
For a solo founder, that's tempting. But before handing over a recurring task, I'd want to know: when a step fails overnight, does it recover, stop, or quietly keep going?
@RickDev3000@dotnet With each agent in its own worktree, I'd test a file changing after checks pass but before completion is recorded. Is the pass bound to the exact tree that was checked, or can a later edit still inherit a green result?
@__padmanabhan @neatlogs The before/after traces are useful. For the research agent, one case I’d test is two sources disagreeing on a number. Does “same answers” also check that citations still support each claim after the five fixes?
@TheMaxPrain Separate computers and memory make the parallel setup interesting. One case I’d test is an agent losing its cloud computer halfway through a delegated task. Does the dashboard preserve its progress so another agent can resume without repeating completed work?
@AakarshDhi43154@neatlogs The recovery trace is revealing. One case I’d test for Spend Radar is a peer with missing ML data but valid sales data, followed by an empty fresh sample. Does recovery preserve the peer and mark ML as unknown?