Tech Dive

helping engineers understand how systems scale, fail & recover in production.

[Tech Dive @techhdive]

@CKPillaiAI great addition. having explicit stopping criteria is necessary to avoid continuing the loop unnecessarily.

[Tech Dive @techhdive]

when building ai agents, some tasks require multiple operations like searching, reasoning and validating previous information.

in that case, you create a loop where the LLM keeps interacting with the application until it has enough information to produce the final answer.

but here's the problem.

you need to maintain the state of the current task like

- what has been done so far
- observations that have been made
- whats still left
etc.

if you don't maintain this state, the model will repeat previous operations, lose track of what has already been done or make decisions without considering previous findings.

so in every iteration, you maintain the task state and send the relevant context back to the model.

this allows the model to continue from where it left off instead of treating every iteration as an independent query.

the loop continues until the model has enough information to produce the final answer.

[Tech Dive @techhdive]

as soon as you wake up, do 2 hours of focussed work with your coffee.

it just straightforward, effective and easy to follow.

you can get many things done, which you feel imp.

[Tech Dive @techhdive]

mcp servers help LLMs get additional information and generate better answers.

but for that, the model needs to decide which tool to call for a given query.

recently, i faced an issue where the model wasn't able to make the right tool calls because there were too many tools with overlapping use cases.

it kept making wrong calls, which led to wrong answers.

after looking into it, i added clear instructions on when to use which tool and started providing only the tools needed instead of everything an mcp server returns.

this helped the model to choose the right tools.

[Tech Dive @techhdive]

turning down people asking you for money is quite hard.

a relative asked me for money for the first time. i gave it to him, then had to bug him 7 or 8 times just to get it back.

then he came asking for money again. i refused, but he insisted he'd return it in 2 days.

it's been a year since then.

now i don't pick up his calls. if i ever run into him in person, i'm going to tell him the reason to his face.

he only ever called when he needed money.

[Tech Dive @techhdive]

it's more about the exposure your job provides you. if your job doesn't require these things, it's hard to learn them through work.

so build a project that actually requires you to learn rag, tool calls, mcp, agent memory, etc. it's the right time to do this, with so many products trying to build ai agents for their use cases.

[Tech Dive @techhdive]

@vineerpasam the probability of getting shortlisted in the first place when you don't know rag, vector databases and ai agents is also minimal. almost every other job description you'll see mentions these.

[Tech Dive @techhdive]

you use embedding models to convert text into vectors.

to store these vectors for fast retrieval, you can use chroma db or postgres with the pgvector extension.

when a user asks a question, you convert the query into a vector using the same embedding model and compare it against the stored vectors to find the closest matches.

these matches can then be used as relevant context for the LLM before it generates a response.

that's the basic idea behind retrieval augmented generation (RAG)

[Tech Dive @techhdive]

@buswe_com 💯

[Tech Dive @techhdive]

when you give an LLM access to tools, you provide tool contracts that describe what each tool does, what arguments it accepts and what it returns.

but that's not all.

you can also add a system prompt that clearly instructs the model on when to use which tool and when not to use any tool at all.

this helps make tool selection more consistent and predictable.

[Tech Dive @techhdive]

at a high level, building an ai agent comes down to a few things.

- you need a model that can understand user queries and generate responses.

- use tools to let the model access external data and perform actions.

- give clear instructions on how the agent should behave and when to use tools.

- create a loop to handle multiple conversation turns and tool calls.

- store conversation history and pass relevant messages to the model so it has the context.

- use mcp when you want a way to connect your application to external tools and context sources.

- build a rag pipeline if you want the model to retrieve relevant information from your own data.

that's the high level overview of building an ai agent.

each of these points can go much deeper, but this is a good starting point.

[Tech Dive @techhdive]

keep models, api endpoints and other configurable settings separate from your application code.

this makes it easier to switch models, change configurations or use different apis without modifying the codebase.

keep api keys in environment variables or a secrets manager, not directly in your code.

it makes your application easier to maintain and configure.

[Tech Dive @techhdive]

writing when learning is really helpful. i am not talking about taking notes.

writing while learning, when your pen just moves with what you're thinking, helps you focus more and actually grasp what you're learning.

it's just a personal observation.

[Tech Dive @techhdive]

@HedgeCollective correct, it basically expands the information available to the model for your use case.

[Tech Dive @techhdive]

LLMs don't have access to real time data.

if you ask a model, "how's the weather in new york today?"

it can't answer with current weather information using only what it learned during training.

to solve this, you can build a tool in your application and make it available to the model.

for example, you can create a python function that calls a weather api and returns the latest weather information.

when you ask about the weather in new york, the model can decide to call that tool, receive the result and use it to generate an answer.

the same concept applies to web search, rolling a dice, reading files or performing other operations.

LLMs can reason and generate answers, but tools allow them to access external information and perform actions beyond their built in capabilities.

[Tech Dive @techhdive]

@Bsoutherland23 yeah, this will work for info available on the web, but for any internal documents or data, you'd still need a tool.

[Tech Dive @techhdive]

if you appear for an interview and are asked to code in an LLD round, how would you handle it if you've been using ai to write code for months?

[Tech Dive @techhdive]

@buswe_com yes 💯

[Tech Dive @techhdive]

install ollama using brew.

run:

- ollama pull qwen3:4b

- brew services start ollama

you now have a local model ready to use.

it's great for learning how to integrate ai models into your applications, without paying for api usage.

[Tech Dive @techhdive]

models can reason and generate answers, but they can't directly read files or access live information unless your application provides the right tools.

for example, say you ask a model to read a file from your system.

the model can't just open the file on its own. it needs a tool that can read files.

but how does the model know that such a tool exists?

your application provides the model with a description of the available tools and how to use them.

now, when you ask it to read a file, the model can identify that it needs the read file tool and request its execution.

your application validates the request, executes the tool and sends the file content back to the model.

the model then uses that content to answer your question.

this is how tools help models do things they can't do on their own.

the model decides when a tool is needed and requests its execution. your application executes the tool and sends the result back to the model.