AI pilots rarely stall because of the model. Usually:
1. No agreed success metric or baseline
2. The last 20% of cases was never designed
3. No plan for "what happens when it's wrong?"
A pilot asks "can it work?" Production asks "what do we do when it doesn't?"
Next is the case generator. The first data release is planned with 2,000 training cases, 500 public test cases and 300 hidden ones that we keep for verified runs. The cases are synthetic, built from the official rules. https://t.co/9qcz0jXBDo
The first release of our Decision Benchmark is up. For now it's the spec, the tax rules and the output format for supplier-invoice decisions in Malaysia and Singapore. The cases themselves follow once the generator is built.
We also caught two mistakes of our own while reviewing the plan for the generator. The tax sheet never said what Singapore's GST rate is (9%). And our repo check blocks all .jsonl files, so it would have rejected every results submission. The fix is planned.
Agent keeps repeating a mistake? I'd rather refactor the code to be more agent-ready than add another line to the prompt. Lauren Tan made a similar case at AI Engineer CODE (11 Nov): fix it in the code or a check, not in AGENTS.md. Which do you reach for?
@MTSlive@walden_yan What makes a codebase safe for an agent to merge into: test coverage, module boundaries, something else? My bet is refactoring the code so an agent can change it correctly first time beats tuning prompts.
@vishwanath_tns Smaller diffs help, but I’d cut what needs a human at all: let AI review first so the human only sees what’s left, and refactor the code so an agent can change it correctly in the first place.
Nearly 70% of Thai organisations treat AI mainly as IT's job, and 40% can't find a use case (Deloitte Thailand, 2026). Those are linked. The best first use case sits with the team that decides the same thing every day: finance, ops, claims.
The first number I check on a document agent is the handover rate: the share of cases it passes to a person. Near zero means it is guessing on cases it should not touch. Too high means nothing got automated. Accuracy only means something next to it.
A grant changes who pays, not whether the project works. EDGE can fund up to 50% of AI on a document process, but only if you apply before work starts or a deposit is paid. Measure, scope, apply. https://t.co/VcCXD10AwT
I'm shutting down Kreye, my AI-powered canvas workspace.
Not a failure story — a quiet one. It worked, people used it, but activity stayed flat while the infra bills didn't. I'd rather point that time at projects I'm more excited about.
So I'm doing the honest thing: saying thanks, and open-sourcing both repos on the way out.
→ schema-driven widget renderer
→ ReactFlow canvas w/ semantic linking
→ FastAPI backend w/ auth + Stripe + AI orchestration
Full retrospective + code:
https://t.co/LBLpzXaBX3