SpaceX’s massive corpus of world-class engineering data (excluding material blocked by ITAR) will be added during supplemental training of the 2T run.
This will dramatically improve Grok’s engineering capabilities.
I’m currently using Grok 4.5 99% of the time now for both planning and tasks.
The fact that I don’t have to switch out to a cheaper model to do tasks means there is less context window model switching and faster speed.
Feels incredible.
Tesla's Robotaxi/ride-hailing service is now available in 7 areas:
• Orlando: Unsupervised Model Y rides only (new)
• Miami: Unsupervised Model Y rides only
• Tampa: Unsupervised Model Y rides only (new)
• Dallas: Unsupervised Model Y rides only
• Houston: Unsupervised Model Y rides only
• Austin: Mix of Unsupervised & safety monitor Model Y rides
• Bay Area: Safety monitor Model Y rides only
Someone tipped me off to a website where it compiles all the real ratings for movies on Rotten Tomatoes and compares it to the fake push ratings they try to portray as real ratings. Really eye opening.
Will link the website in first reply but it's called PopcornGap.
Backpropagation by hand ✍️ ~ 11 steps walkthrough below
Backpropagation is the algorithm that actually trains a neural network, and it is where most people stop following along. It is not calculus you cannot do. It is matrix multiplication, working backward, one layer at a time.
So I drew and calculated one entirely by hand.
Goal: push the loss gradient back through a 3-layer network and land on a new value for every weight and bias.
= 1. Given =
A 3-layer perceptron, an input X, predictions Ypred = [0.5, 0.5, 0], and the truth Ytarget = [0, 1, 0].
= 2. Backprop gradient cells =
Let us draw empty cells for every gradient we are about to compute. The shape of the answer comes first.
= 3. Layer 3 softmax =
We get dL/dz3 straight from Ypred minus Ytarget = [0.5, -0.5, 0]. No chain rule needed, and that shortcut is the whole reason softmax and cross-entropy are paired.
= 4. Layer 3 weights and biases =
Let us multiply dL/dz3 by [a2 | 1]. One multiplication gives the gradient for W3 and b3 together.
= 5. Layer 2 activations =
We multiply dL/dz3 by W3 to get dL/da2. The gradient moves back across a layer the same way the signal moved forward.
= 6. Layer 2 ReLU =
Let us pass it through the gate: keep the gradient where the activation was positive, zero it everywhere else.
= 7. Layer 2 weights and biases =
We multiply dL/dz2 by [a1 | 1]. The same figure as step 4, one layer up.
= 8. Layer 1 activations =
Let us multiply dL/dz2 by W2.
= 9. Layer 1 ReLU =
We apply the same gate again, now on a1.
= 10. Layer 1 weights and biases =
Let us multiply dL/dz1 by [x | 1], and every weight in the network now has a gradient.
= 11. Update =
We subtract, and the network has learned. In practice a learning rate scales this step.
The gradients:
dL/dz3 = [0.5, -0.5, 0]
dL/da1 = [1, -2, 2, -1]
dL/dz1 = [0, -2, 2, -1]
The takeaway: matrix multiplication is all you need. Just like the forward pass, backpropagation is matrix multiplications end to end. You can do every one by hand, slowly and imperfectly, which is exactly why a GPU's ability to do them fast mattered so much to deep learning.
💾 Save this post!
One pattern I find useful for working with LLMs is a nice long ramble session. Sometimes the LLM needs more bits to understand what you're trying to achieve, but you're too lazy to type them. In these cases I like to lean back, switch to /voice and just ramble for like 10 minutes, total mess, anything goes, full stream of consciousness. Sometimes I declare it up top, something like "switching to speech recognition sorry for any typos...". Sometimes I turn it into a small interview of a few turns. But I find that the LLMs are somehow very good at reconstructing long incoherent rambles and often their echo of your own tangle of thoughts comes out quite a bit cleaner than what you started with. The result is that you improve the mind meld and have to correct things less from that point on.
Rate of improvement is accelerating.
Users should notice a meaningful improvement in the usefulness of the Grok Build harness with our V9 foundation model (aka Grok 4.5) every week.
“That was fun, I’ll do it again when I get out,” said the 22-year-old Gambian violent offender after he stabbed a 55-year-old Italian man twenty times.
And he will be released again. Perhaps to succeed in killing a European next time.
My husband and I are honored and thrilled to participate in the Invest America program and gift a share of our SpaceX stock to a Trump Account for each of more than two million children across our great nation.
Every American child under 18 can benefit from having a Trump Account, but our gift specifically goes to the Accounts of children (ages 11-17) that live in areas with lower average household incomes with a bit more emphasis for those that live near our central Texas home.
We have been fortunate in our careers and hope this gift encourages the next generation to continue the journey of enabling humanity to live and fly amongst the stars.
A huge thanks to all that worked to bring this incredible program to life.
One of Elon Musk’s companies, Tesla, is making cars for blind people.
Another one of his companies, Neuralink, is working on restoring sight to the blind.