Kubernetes Components explanation:
📦 Pods → Where your application runs
🖥️ Nodes → Machines that host Pods
🌐 Cluster → Collection of Nodes managed together
🔄 Deployment → Ensures your application stays updated and available
👥 ReplicaSet → Maintains the desired number of Pods
💾 StatefulSet → Designed for databases and stateful applications
⚙️ DaemonSet → Runs one Pod on every Node (great for monitoring/logging agents)
✅ Job & CronJob → One-time and scheduled tasks
🌍 Service → Gives Pods a stable network identity
🚪 Ingress → Routes external traffic to your applications
🔐 ConfigMap & Secret → Manage configuration and sensitive data securely
💽 PV & PVC → Persistent storage for applications
🏷️ Namespace → Organize workloads across teams or environments
🤖 Kubelet → Agent running on every Node
🔀 Kube-Proxy → Handles networking between services and Pods
🎯 Scheduler → Decides where Pods should run
🧠 API Server → The heart of Kubernetes communication
⚡ Controller Manager → Continuously keeps the cluster in its desired state
💡 Think of Kubernetes like a city:
🏙️ Cluster = City
🏢 Nodes = Buildings
🏠 Pods = Apartments
👨💼 Deployment = Property Manager
🚦 Service = Reception Desk
🛣️ Ingress = Main Entrance Gate
📦 PV/PVC = Storage Room
8 RAG architectures for AI Engineers:
(explained with usage)
1) Naive RAG
- Retrieves documents purely based on vector similarity between the query embedding and stored embeddings.
- Works best for simple, fact-based queries where direct semantic matching suffices.
2) Multimodal RAG
- Handles multiple data types (text, images, audio, etc.) by embedding and retrieving across modalities.
- Ideal for cross-modal retrieval tasks like answering a text query with both text and image context.
3) HyDE (Hypothetical Document Embeddings)
- Queries are not semantically similar to documents.
- This technique generates a hypothetical answer document from the query before retrieval.
- Uses this generated document’s embedding to find more relevant real documents.
4) Corrective RAG
- Validates retrieved results by comparing them against trusted sources (e.g., web search).
- Ensures up-to-date and accurate information, filtering or correcting retrieved content before passing to the LLM.
5) Graph RAG
- Converts retrieved content into a knowledge graph to capture relationships and entities.
- Enhances reasoning by providing structured context alongside raw text to the LLM.
6) Hybrid RAG
- Combines dense vector retrieval with graph-based retrieval in a single pipeline.
- Useful when the task requires both unstructured text and structured relational data for richer answers.
7) Adaptive RAG
- Dynamically decides if a query requires a simple direct retrieval or a multi-step reasoning chain.
- Breaks complex queries into smaller sub-queries for better coverage and accuracy.
8) Agentic RAG
- Uses AI agents with planning, reasoning (ReAct, CoT), and memory to orchestrate retrieval from multiple sources.
- Best suited for complex workflows that require tool use, external APIs, or combining multiple RAG techniques.
Most architectures here involve some form of retrieval-time decision. But they all run on top of whatever was already indexed.
If that indexing step outputs messy chunks, every architecture inherits them. Improving it is a separate problem from the 8 above.
I wrote about a better unit for the indexing step. The technique:
- cuts corpus size by 40x.
- reduces tokens per query by 3x.
- improves vector search relevance by 2.3x.
And it doesn't alter the retrieval algorithm, the reranker, or the embedding model.
Read it in the article quoted below.
I create a background job to process Outbox messages.
This is the simplified version, without error handling. But you should take care of that because handling a message can (and will) fail.
The job processes the messages in batches on a recurring schedule. 👇