Nassim · AI Agents

AI Automation Architect · n8n workflows & agents that don't fail silently · tests, CI, measured results · Code on GitHub ↓

[Nassim · AI Agents @NassimAIx]

First LLM eval in n8n.
20 labeled cases. Run 1: 18/20.

Bug 1: max_tokens too low. Model spent it all thinking, returned nothing. No error.
Bug 2: it calculated a total nobody asked for.

Fixed. 3 runs: 20/20. Then 10 unseen: 10/10.

Agents don't fail loudly. They fail politely.

[Nassim · AI Agents @NassimAIx]

@DanielSmidstrup @n8n_io The useful split is reversible vs irreversible actions. Reading and drafting can run freely, but cancel, pay or send should stop at a confirm step. The part I'd check is what happens when the permission prompt itself times out: it should default to "no", not to "continue".

[Nassim · AI Agents @NassimAIx]

@tsvillain The "advancing" guard is the part I'd look at twice: if recordCallTurn throws, the flag stays true and the call goes silent, because nothing resets it. A try/finally, or a fallback that plays the next fixed question anyway, keeps one failed write from freezing the interview.

[Nassim · AI Agents @NassimAIx]

Added email alerts to my n8n agent runner.

Rule: alert only on real failures, never on a bad request (4xx), or you learn to ignore alerts.

Tested 3 cases: ok → 0 mail, empty → 0 mail, service down → 1 mail with cause + link.

[Nassim · AI Agents @NassimAIx]

@SachinSidd50843 "In production, not demos" is a good filter. The question I'd ask any builder here: what was the first failure you only found once real users hit it? The silent ones are the interesting ones, where every step reports success and the output is still wrong.

[Nassim · AI Agents @NassimAIx]

@DanielSmidstrup Anything that writes somewhere I can't easily inspect afterwards: sending an email, deleting, touching billing. A wrong read is cheap, a wrong write isn't, so the check I'd want sits on the side effect, not on the reasoning.

[Nassim · AI Agents @NassimAIx]

@webjuice_ie Review-and-approve is the right gate. What I'd log per edit is the diff plus the client message that triggered it, so a rollback and "who asked for this" are one lookup, and a misread request gets caught in review instead of live.

[Nassim · AI Agents @NassimAIx]

@ConsciousRide Easy to miss in evals, easy to reproduce in a test: merge the record between the lookup and the write, then assert the agent aborts instead of acting. I'd add that failure test to any agent that touches customer data.

[Nassim · AI Agents @NassimAIx]

@mattpocockuk A ubiquitous language also gives you something testable: if the glossary is a file, a plain script can flag any term in the agent's output that isn't in it. The agent then drifts visibly instead of quietly renaming things.

[Nassim · AI Agents @NassimAIx]

@svpino @n8n_io 98% completion is the number I'd read next to the failure split: of the 2%, how many stopped and asked vs acted wrongly? For a browser agent that can cancel and pay, that tail matters more than the headline rate.

[Nassim · AI Agents @NassimAIx]

Weekend on a Wednesday, somewhere in the countryside. Tested exactly-once delivery on my dog: ball thrown, ball returned, once per throw. A system that never fails silently. 🐕

[Nassim · AI Agents @NassimAIx]

@n8n_io Marking a stuck step uncertain and checking the account first is the right order: a timed-out cancel may have gone through, and a blind retry means a double action. What does Nod do when the account itself is ambiguous, like a cancellation pending until period end?

[Nassim · AI Agents @NassimAIx]

@paschals_robert @m_t_ramkrushna The slow tail is the catch: a lease sized for the typical call expires on the outlier. I'd renew it with a heartbeat while the call runs, so the lease tracks whether the worker is alive, not how long it takes. Do you extend leases like that, or size for the slowest case?

[Nassim · AI Agents @NassimAIx]

@cneuralnetwork Cheap runs shift the bottleneck to eval: when training costs dollars, the expensive part is knowing a run is bad early. Fixed-interval evals on a held-out set plus a kill rule catch a diverging run in the first hour, not at the end.

[Nassim · AI Agents @NassimAIx]

@HelloVyom @inkbox_ai Multi-channel booking is where double reservations come from: if the call drops mid-confirmation, a retry can book twice. I'd keep one booking ID as the source of truth and check it across iMessage, WhatsApp and email before any retry.

[Nassim · AI Agents @NassimAIx]

@alxdanslebueno Le mélange vient souvent du contexte partagé entre les missions. En plus d'une mission/un livrable, écris avant de lancer un critère de réussite vérifiable (« le fichier X contient Y ») : tu contrôles en 10 secondes au lieu de relire tout le résultat.

[Nassim · AI Agents @NassimAIx]

@JinjingLiang Cheap passes on CI polling hold up until the status is ambiguous: a flaky retry or partial log gets read as green. Route anything that isn't a clean pass/fail to a higher-effort rerun and track the escalation rate per task type, so the savings don't hide silent misses.

[Nassim · AI Agents @NassimAIx]

@paschals_robert @m_t_ramkrushna The lease expiry is what frees a key after a crashed worker. I'd pair it with a reconcile pass that asks the downstream system what really happened before any retry, so a call that went through never runs twice. Lease length per call type, or one default?

[Nassim · AI Agents @NassimAIx]

@m_t_ramkrushna One catch: after the lease expires, "no result yet" must be its own state, not "failed". Otherwise an ambiguous timeout gets retried and double-fires. A way to model it: three outcomes, done, not done, can't tell. The last one goes to a safe read-back or a human.

[Nassim · AI Agents @NassimAIx]

https://t.co/3NP4RssnWV

[Nassim · AI Agents @NassimAIx]

Khirami Agent: every recommendation must trace to a diagnosed root cause. A plain-Python guard flags orphans, missing owners, missing done-conditions. No model in the check.

333 tests, 332 offline on a stubbed model. Answer quality: not measured yet. Written in the README.