M.T.Ramkrushna (MTR)

Applied AI Engineer: LLM systems, RAG, agents. Co-founder @ Replynlink, turning Instagram comments into DMs. Weekly AI posts + practical tips.

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

This week in AI, for builders:

• OpenAI + Anthropic + DeepMind on shared frontier safety standards
• OpenAI phased text watermarking (EU AI Act)
• Cohere North 2: agent memory + harness

Theme: standards, provenance, and memory around the agent.

#AI #LLM

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

@0xPascual Agreed that the bottleneck moved to testing and debugging workflows. Replaying failed production traces as regression tests has been the most useful step for me.

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

@surendranb @ArizePhoenix Pairing deterministic checks with an LLM judge in a weekly loop is a solid setup. I'd keep the judge prompt versioned too, so a score change can't come from the judge drifting.

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

@Harry_The_Nerd Tokens also decide what the model forgets, not just the bill. One habit that helped me: log token counts per prompt section so you can see which part of the context is eating the budget.

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

Every LLM prompt with a support ticket in it is customer PII leaving your system. So is your prompt log.

Presidio finds names, emails, cards & your IDs, then swaps them for <PERSON> before the call.

Tested: address & passwords slip through. Layer it.

https://t.co/cDxFYdHHCf

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

@tsvillain The explicit Done signal makes turn boundaries deterministic, and the 60-second timer is a sensible recovery path. I'd compare answer completeness and candidate drop-off against the old detector so the UX trade-off is measured alongside transcript accuracy.

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

@Lawrence_Njobo Replacing free-form text with a probability for binary decisions can eliminate a lot of brittle parsing. I'd still check calibration by task type and keep a thresholded fallback for ambiguous cases—does the API expose calibration metrics?

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

@curiousjar Right, the prompt is a request and the architecture is the enforcement. That's why I'd keep the planner away from untrusted text and gate every tool call with permissions checked in code, not in the prompt.

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

You can't prompt your way out of prompt injection.

GPT-4o mini, with instruction hierarchy, still let 276 AgentDojo attacks through.

Separate planning from untrusted data and Gemini 2.5 Pro went from 300 to 0.

Fix the architecture, not the prompt.

https://t.co/gKwCRiaT9P

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

Your Python function is one decorator away from being a tool Claude & Cursor can call.

FastMCP: @mcp.tool turns type hints into the MCP schema, validates inputs, and you test it with fastmcp call before any LLM touches it.

Keep tools narrow.

How-to:
https://t.co/JSHqehLdhN

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

@rohanpaul_ai That distinction matters: token volume is activity, not delivery. I’d pair spend with accepted task completion and rework rate, then segment by task complexity so teams don’t reward verbosity.

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

@curiousjar Nice analogy for grounding the concept. In production, I’d also teach the skill when to stop and hand control back; a checklist that includes tool permissions and a verification step keeps the manual useful under edge cases.

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

@saadkhalidhere The canary test is a neat, cheap check for retrieval leaks. I'd also vary the canary phrasing and chunk boundaries, since a leak can show up as a paraphrase rather than a verbatim return.

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

@vatsapranav_ Fully agree that the eval set is the spec. Writing those 20 input/output pairs first also exposes where the requirements are vague. Do you version the set alongside the prompts?

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

@JesseNotAi Exactly—restoring the agent is only safe if its permissions and connected resources return to the approved state too. I’d snapshot the policy/config version with each checkpoint and require human review if live grants have drifted.

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

@NassimAIx @paschals_robert Per-call-type is my preference, with a heartbeat for long-running calls and a hard upper deadline. Reconciliation before replay is the safety check that keeps an ambiguous timeout from turning into a duplicate side effect.

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

Your AI agent will call the same tool twice.

Timeout. Retry. First call worked. Now: two refunds.

Payments teams fix it:
• stable key per action (run+step/call_id)
• store the first result, replay it
• return "already done" to the model

https://t.co/hX7aS0lWIb

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

@paschals_robert @NassimAIx That’s exactly the overlap a short lease can create: the retry may start while the first worker is still running. Per-call-type leases, heartbeats, and a stable downstream idempotency key give you safer recovery without guessing one timeout for everything.

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

@NassimAIx @paschals_robert Heartbeats seem like the better fit here: they measure worker liveness without penalizing legitimate long calls. I’d add a fencing token so a worker that resumes after lease loss can’t commit stale results.

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

@paschals_robert @NassimAIx Exactly—renewal tracks liveness rather than trying to guess the slowest call. I’d pair the heartbeat with a hard deadline and fencing token, so an expired worker can’t commit late.

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

@adrianogomes @ReignSwords @InfoQ That file-based plan makes approval auditable, and re-reading it before each chunk gives the agent a stable contract. I’d also tie each accepted plan version to its diffs so later changes stay traceable to what was approved.

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

@adrianogomes @ReignSwords @InfoQ Plan-first then small diffs is a solid pattern. Do you have the agent write the plan to a file so the approval step is reviewable, or is it inline in the session?

[M.T.Ramkrushna (MTR) @m_t_ramkrushna]

@Galo2027 Repeatable beats clever, agreed. Checking the reusable skill/instructions into the repo with a few eval cases also lets you catch regressions when the model changes.