One pattern I find useful for working with LLMs is a nice long ramble session. Sometimes the LLM needs more bits to understand what you're trying to achieve, but you're too lazy to type them. In these cases I like to lean back, switch to /voice and just ramble for like 10 minutes, total mess, anything goes, full stream of consciousness. Sometimes I declare it up top, something like "switching to speech recognition sorry for any typos...". Sometimes I turn it into a small interview of a few turns. But I find that the LLMs are somehow very good at reconstructing long incoherent rambles and often their echo of your own tangle of thoughts comes out quite a bit cleaner than what you started with. The result is that you improve the mind meld and have to correct things less from that point on.
Backpropagation by hand ✍️ ~ 11 steps walkthrough below
Backpropagation is the algorithm that actually trains a neural network, and it is where most people stop following along. It is not calculus you cannot do. It is matrix multiplication, working backward, one layer at a time.
So I drew and calculated one entirely by hand.
Goal: push the loss gradient back through a 3-layer network and land on a new value for every weight and bias.
= 1. Given =
A 3-layer perceptron, an input X, predictions Ypred = [0.5, 0.5, 0], and the truth Ytarget = [0, 1, 0].
= 2. Backprop gradient cells =
Let us draw empty cells for every gradient we are about to compute. The shape of the answer comes first.
= 3. Layer 3 softmax =
We get dL/dz3 straight from Ypred minus Ytarget = [0.5, -0.5, 0]. No chain rule needed, and that shortcut is the whole reason softmax and cross-entropy are paired.
= 4. Layer 3 weights and biases =
Let us multiply dL/dz3 by [a2 | 1]. One multiplication gives the gradient for W3 and b3 together.
= 5. Layer 2 activations =
We multiply dL/dz3 by W3 to get dL/da2. The gradient moves back across a layer the same way the signal moved forward.
= 6. Layer 2 ReLU =
Let us pass it through the gate: keep the gradient where the activation was positive, zero it everywhere else.
= 7. Layer 2 weights and biases =
We multiply dL/dz2 by [a1 | 1]. The same figure as step 4, one layer up.
= 8. Layer 1 activations =
Let us multiply dL/dz2 by W2.
= 9. Layer 1 ReLU =
We apply the same gate again, now on a1.
= 10. Layer 1 weights and biases =
Let us multiply dL/dz1 by [x | 1], and every weight in the network now has a gradient.
= 11. Update =
We subtract, and the network has learned. In practice a learning rate scales this step.
The gradients:
dL/dz3 = [0.5, -0.5, 0]
dL/da1 = [1, -2, 2, -1]
dL/dz1 = [0, -2, 2, -1]
The takeaway: matrix multiplication is all you need. Just like the forward pass, backpropagation is matrix multiplications end to end. You can do every one by hand, slowly and imperfectly, which is exactly why a GPU's ability to do them fast mattered so much to deep learning.
💾 Save this post!
Someone tipped me off to a website where it compiles all the real ratings for movies on Rotten Tomatoes and compares it to the fake push ratings they try to portray as real ratings. Really eye opening.
Will link the website in first reply but it's called PopcornGap.
Tesla's Robotaxi/ride-hailing service is now available in 7 areas:
• Orlando: Unsupervised Model Y rides only (new)
• Miami: Unsupervised Model Y rides only
• Tampa: Unsupervised Model Y rides only (new)
• Dallas: Unsupervised Model Y rides only
• Houston: Unsupervised Model Y rides only
• Austin: Mix of Unsupervised & safety monitor Model Y rides
• Bay Area: Safety monitor Model Y rides only
I’m currently using Grok 4.5 99% of the time now for both planning and tasks.
The fact that I don’t have to switch out to a cheaper model to do tasks means there is less context window model switching and faster speed.
Feels incredible.
SpaceX’s massive corpus of world-class engineering data (excluding material blocked by ITAR) will be added during supplemental training of the 2T run.
This will dramatically improve Grok’s engineering capabilities.
A Cybercab with no steering wheel, a massive display, and high speed Starlink is going to be the best place to work or watch a movie as you take a ride across town
One very useful tool in Agent mode is image cropping of Grok Imagine .
I don't know why no one talks about it, as it is a crucial element for creating smooth transitions.
During a conversation, you need to frame each person and switch camera angles to add energy to your video.
Being able to pick the frame you want and crop it saves a huge amount of time.