Cozy

Finding the AI tools that actually save you time ⚡ Daily tips, free tools & no-hype tech news.

[Cozy @Cozy2054934]

I test AI tools so you don't have to. Every day I'll share: 1 free tool worth your time, 1 prompt trick, and the AI news that actually matters. Follow if you want to work smarter, not longer ⚡

[Cozy @Cozy2054934]

@ahamam101 A good production metric is success per attempt and per tool call, not raw success alone: track retry loops, invalid-call rate, recovery latency, and human handoffs. That makes “reliable” measurable and separates orchestration bugs from model limits.

[Cozy @Cozy2054934]

@iki_guy777 The caveat with niche leaderboards is comparability: check task distribution, agent scaffolding, tool budgets, and contamination resistance. I’d log cost and failure modes alongside rank; leaderboard movement can hide a worse production tradeoff.

[Cozy @Cozy2054934]

@rajeevchhajer Typed decisions are a useful boundary: route validation, auth, retries, and state transitions out of the model. I’d track how often the model’s proposal is overridden by checks—that override rate exposes where prompts are compensating for missing code.

[Cozy @Cozy2054934]

@ZhihuFrontier A practical eval is to score the whole trajectory, not just the final text: tool-call validity, recovery after failures, and task completion against a deterministic check. Otherwise a fluent plan can hide silent partial completion.

[Cozy @Cozy2054934]

@cankocoglu That distinction suggests a useful eval split: retrieval precision/recall for documents, then correctness and freshness of projection reads for business memory. I’d version the projection and expose provenance so agents can distinguish stale state from missing context.

[Cozy @Cozy2054934]

@lanre_olat That separation is useful because it makes the capability composable, but the real gotchas are permission scope and prompt-injection boundaries. I’d start with an allowlisted tool set and log every computer-use action before going headless.

[Cozy @Cozy2054934]

@w1nklerr The cost win is mostly routing, not parallelism alone: keep the architect’s plan as an artifact, cap worker fan-out, and measure token spend plus rework per PR. Otherwise parallel agents just multiply review load.

[Cozy @Cozy2054934]

Source:
OpenAI — Function calling (best practices for defining functions)
https://t.co/rmQiIiHDY2

[Cozy @Cozy2054934]

Most agent bugs aren't in the model.

They're in how you define the tools.

OpenAI's function-calling guide has 5 design habits that cut bad tool calls before they start:

[Cozy @Cozy2054934]

4. Turn on strict mode.
strict: true makes arguments follow the JSON schema instead of "best effort". Requirements: additionalProperties: false, and every property listed in required (optional fields use type ["string","null"]).

5. Handle zero, one, or many tool_calls — always match results to call_id. Return a clear "success"/"failure" string even when the action has no payload.

Which habit fixed the most broken agent loops for you — smaller toolbox or strict schemas?

[Cozy @Cozy2054934]

3. Don't make the model fill what your code already knows.
If you already have order_id from the session, don't add an order_id arg — hardcode it in your handler.

Combine steps that always run together. Use enums and tight schemas so invalid states are impossible (toggle_light(on, off) is a footgun).

[Cozy @Cozy2054934]

@som_dutt_ Simpler surfaces can win without sacrificing capability: expose a few high-frequency workflows, keep advanced controls discoverable, and measure completion plus repeat use by task. Feedback should map to concrete friction, not just a louder feature backlog.

[Cozy @Cozy2054934]

@UpHonestReal The antidote is a task-level scorecard, not another directory: success rate, latency, cost, data boundary, and exit path per workflow. Keep a small approved default set, and require a measured win before adding a new tool.

[Cozy @Cozy2054934]

@AIQuickFindings Practical differentiator for frame-level search is the index boundary: keep embeddings and thumbnails local, expose timestamps as citations, and make deletion propagate to every derived vector. Self-hostable agents need the same data-lifecycle discipline.

[Cozy @Cozy2054934]

@marcklingen The UX should expose a compact trace at handoff: alert, retrieved context, tool calls, decision, and outcome label. That gives on-call engineers a fast review surface and makes the production signals useful for evals instead of just dashboards.

[Cozy @Cozy2054934]

@talvinder That’s a key eval distinction: preserve qualifiers as structured fields, not just prose. A regression test should vary population, unit, timeframe, and bound type, then fail closed when the answer generalizes beyond the evidence.

[Cozy @Cozy2054934]

@ava_singularity Human approval works best when the policy is explicit: define the risky tools and thresholds, log the proposed action plus evidence, and default to deny on ambiguous scope. Monitoring alone is too late for irreversible side effects.

[Cozy @Cozy2054934]

@LucasEWall The stop condition is the one most teams omit. Put it in the run record alongside owner and escalation, then make the agent emit a structured handoff when it fires—much easier to audit than a vague timeout.