Major cheat code for life: Always assume things will work out, then do the work to make it true. It’s a mindset of active optimism. Earned faith. It creates a quiet confidence that allows you to tolerate the uncertainty of life better than anything else.
@shadowfax_in I’m not getting otp for return pick up on sms or WhatsApp. The delivery partner has tried to send multiple times. im getting other sms but not shadowfax otp. Pls help
Backpropagation by hand ✍️ ~ 11 steps walkthrough below
Backpropagation is the algorithm that actually trains a neural network, and it is where most people stop following along. It is not calculus you cannot do. It is matrix multiplication, working backward, one layer at a time.
So I drew and calculated one entirely by hand.
Goal: push the loss gradient back through a 3-layer network and land on a new value for every weight and bias.
= 1. Given =
A 3-layer perceptron, an input X, predictions Ypred = [0.5, 0.5, 0], and the truth Ytarget = [0, 1, 0].
= 2. Backprop gradient cells =
Let us draw empty cells for every gradient we are about to compute. The shape of the answer comes first.
= 3. Layer 3 softmax =
We get dL/dz3 straight from Ypred minus Ytarget = [0.5, -0.5, 0]. No chain rule needed, and that shortcut is the whole reason softmax and cross-entropy are paired.
= 4. Layer 3 weights and biases =
Let us multiply dL/dz3 by [a2 | 1]. One multiplication gives the gradient for W3 and b3 together.
= 5. Layer 2 activations =
We multiply dL/dz3 by W3 to get dL/da2. The gradient moves back across a layer the same way the signal moved forward.
= 6. Layer 2 ReLU =
Let us pass it through the gate: keep the gradient where the activation was positive, zero it everywhere else.
= 7. Layer 2 weights and biases =
We multiply dL/dz2 by [a1 | 1]. The same figure as step 4, one layer up.
= 8. Layer 1 activations =
Let us multiply dL/dz2 by W2.
= 9. Layer 1 ReLU =
We apply the same gate again, now on a1.
= 10. Layer 1 weights and biases =
Let us multiply dL/dz1 by [x | 1], and every weight in the network now has a gradient.
= 11. Update =
We subtract, and the network has learned. In practice a learning rate scales this step.
The gradients:
dL/dz3 = [0.5, -0.5, 0]
dL/da1 = [1, -2, 2, -1]
dL/dz1 = [0, -2, 2, -1]
The takeaway: matrix multiplication is all you need. Just like the forward pass, backpropagation is matrix multiplications end to end. You can do every one by hand, slowly and imperfectly, which is exactly why a GPU's ability to do them fast mattered so much to deep learning.
💾 Save this post!
Andrew Ng just released a 1-hour course on building agentic knowledge Graphs from scratch:
• 00:00 - Introduction to agentic knowledge Graphs
• 03:07 - Construction of agentic Graphs
• 14:00 - Architecture of multi-agent systems
• 23:00 - Building agentic graphs with Google ADK
• 01:06:03 - Why Graphsare the future of agentic AI
Worth more than 10 articles on loop engineering.
Watch it today, then read how to become a graph engineer in the article below.
FDE is the hottest role in tech.
Here's the 30-day plan to actually become one:
Week 1: Build the loop
- Agent looping, tool usage, guardrails
- Context, memory, and an audit trail
- Checkpoint: one working agent, one task.
Week 2: Make it recover
- JSON schemas, not freeform text
- Failure modes for the 1,000 ways it breaks
- "There's only one way something goes right. A thousand ways it goes wrong."
Week 3: Make it measurable
- Retry logic + a golden dataset for evals
- Test cheaper models on sub-tasks
- Score it on revenue, risk, cost
Week 4: Defend it like an FDE
- The pain points, the architecture, the economics
- Then pitch it to real businesses
Think:
- Week 1 = build
- Week 2 = survive
- Week 3 = measure
- Week 4 = sell
That's the whole ladder.
8 RAG architectures for AI Engineers:
(explained with usage)
1) Naive RAG
- Retrieves documents purely based on vector similarity between the query embedding and stored embeddings.
- Works best for simple, fact-based queries where direct semantic matching suffices.
2) Multimodal RAG
- Handles multiple data types (text, images, audio, etc.) by embedding and retrieving across modalities.
- Ideal for cross-modal retrieval tasks like answering a text query with both text and image context.
3) HyDE (Hypothetical Document Embeddings)
- Queries are not semantically similar to documents.
- This technique generates a hypothetical answer document from the query before retrieval.
- Uses this generated document’s embedding to find more relevant real documents.
4) Corrective RAG
- Validates retrieved results by comparing them against trusted sources (e.g., web search).
- Ensures up-to-date and accurate information, filtering or correcting retrieved content before passing to the LLM.
5) Graph RAG
- Converts retrieved content into a knowledge graph to capture relationships and entities.
- Enhances reasoning by providing structured context alongside raw text to the LLM.
6) Hybrid RAG
- Combines dense vector retrieval with graph-based retrieval in a single pipeline.
- Useful when the task requires both unstructured text and structured relational data for richer answers.
7) Adaptive RAG
- Dynamically decides if a query requires a simple direct retrieval or a multi-step reasoning chain.
- Breaks complex queries into smaller sub-queries for better coverage and accuracy.
8) Agentic RAG
- Uses AI agents with planning, reasoning (ReAct, CoT), and memory to orchestrate retrieval from multiple sources.
- Best suited for complex workflows that require tool use, external APIs, or combining multiple RAG techniques.
Most architectures here involve some form of retrieval-time decision. But they all run on top of whatever was already indexed.
If that indexing step outputs messy chunks, every architecture inherits them. Improving it is a separate problem from the 8 above.
I wrote about a better unit for the indexing step. The technique:
- cuts corpus size by 40x.
- reduces tokens per query by 3x.
- improves vector search relevance by 2.3x.
And it doesn't alter the retrieval algorithm, the reranker, or the embedding model.
Read it in the article quoted below.
Major cheat code for life: Stop dragging yesterday into today. The argument. The mistake. The missed chance. It’s already gone. Stop reliving it. Life happens in the direction you face.
Major cheat code for life: The ability to recognize things that don’t matter. The world will pressure you to care about every single thing. To chase every problem. To take every slight personally. To have opinions on everything. Reject that trend. Focus on a few, ignore the rest.
Traditionally, Grandmothers gave babies a warm oil massage before bathing, then gently moved the arms and legs while washing.
Research suggests infant massage promotes relaxation and caregiver–infant bonding, while these movements gently mobilize joints without forcing a stretch.