Andrej Karpathy:
"Prompting is going away.
Delete everything, keep Graph."
In 1 hour he shows how to build Graphs, and why it's the only thing that will be left standing at the end
LLMs → Prompts → Agents → Graphs
a prompt gets you one answer and forgets the job
an agent finishes the job and forgets what it learned
a graph keeps every finished job and turns it into the starting point for the next one
watch it today
then save the full guide on Graphs below before everyone catches up ↓
Every gradient in deep learning is matrix calculus. Almost nobody who ships models can do it by hand.
Autodiff made that survivable, not unnecessary. The moment a loss surface misbehaves, or a Jacobian needs interpreting, you are back to the mathematics.
Magnus wrote the clearest 40 pages ever published on it. Free in my Awesome Math Books list.
Study the math. The framework will change again.
https://t.co/xYRR0bDEnt
I am done with this shit. It is over. The state of engineering right now is horrible. It has been half a month since I started a new role at a big company. Nobody knows anything here. The specs, code, tests, PRDs, tickets, resolution of those tickets, reports, etc., everything is made by Claude Code. Nobody on my team likes this. They are being forced to ship as much as they can. I have heard multiple times from higher management that pushing code is not a bottleneck, so why are we slow? People are working 12 to 13 hours a day just to press enter. Nobody is reading anything. Humans in corporate are doing nothing on their own. Everyone, literally everyone, from an L1 to an L7 engineer here is doing the same thing. Talk to Claude. There is no sense of victory. Nobody is resolving bugs. In reality, nobody is thinking anymore. Everything is done by LLMs. It is so soul-sucking. I would not mind it, to be honest, if we were at least given the time to check out the code and see what is going where. But no, the goal is to just ship. No matter what happens.
this is actually insane
> be tech guy in australia
> adopt cancer riddled rescue dog, months to live
> not_going_to_give_you_up.mp4
> pay $3,000 to sequence her tumor DNA
> feed it to ChatGPT and AlphaFold
> zero background in biology
> identify mutated proteins, match them to drug targets
> design a custom mRNA cancer vaccine from scratch
> genomics professor is “gobsmacked” that some puppy lover did this on his own
> need ethics approval to administer it
> red tape takes longer than designing the vaccine
> 3 months, finally approved
> drive 10 hours to get rosie her first injection
> tumor halves
> coat gets glossy again
> dog is alive and happy
> professor: “if we can do this for a dog, why aren’t we rolling this out to humans?”
one man with a chatbot, and $3,000 just outperformed the entire pharmaceutical discovery pipeline.
we are going to cure so many diseases.
I dont think people realize how good things are going to get
I’m rebuilding GEMM optimisation on H100 from scratch and documenting the full path toward cuBLAS level performance in a deep dive blog.
8 kernels in so far, from naive all the way to Tensor Cores (Async TMA + WGMMA). Just hit 407.7 TFLOP/s, now at 56.9% of cuBLAS with Tensor Cores.
Each kernel is broken down in detail: profiling bottlenecks, PTX and SASS inspection, memory behaviour, and 35+ visuals!!! so far explaining what is happening at the hardware level for each kernel. I also included a full H100 architecture section to ground the optimisation decisions.
Still more to push.
Blog: https://t.co/27pDYyBWPA
Repo: https://t.co/0pTLfDEBk2
pienso mucho en los restos fósiles del COVID: entras a un establecimiento y hay dispensadores de gel hidroalcohólico vacíos, carteles con normas a medio arrancar, sillas con restos de cruces indicando espacios donde uno no se podía sentar…
New fastest shortest-path algorithm in 41 years!
Tsinghua researchers broke Dijkstra’s 1984 “sorting barrier,” achieving O(m log^(2/3) n) time. This means faster route planning, less traffic, cheaper deliveries, and more efficient networks - and a CS curriculum revamp =)
Because instead of being supportive and patient, women generally start to complain that “you’re not the same” and “you’ve changed” and “I’m not happy anymore” and other similar bullshit on top of what we are going through.
It’s extra crap that we shouldn’t have to deal with while trying to solve and deal with everything else.
Women generally don’t help- they throw flaming chainsaws to the knives you’re already juggling.