I regret to inform you that regardless of how broken academia is, you are still responsible for what you become during your PhD.
Your advisor can guide you. Your university can give you a degree. Neither can make you an independent thinker.
Stop chasing paper counts. Stop waiting for the perfect research direction. Stop confusing publications with scientific progress.
Stay curious. Question everything. Take ownership of your research.
The goal was never to become a paper-producing machine. It was to become a scientist.
I'm building a playlist/course for LLM training and inference (WIP).
https://t.co/0JdUqjiiQm
So far, we have common topics like distributed training, fine-tuning, RL, OPD, serving, spec. decoding, and basic optimizers.
What else should I include to make this more complete? 🤔
Anthropic engineers just released the best 90-minute course on building AI agents from scratch I've ever seen:
• 00:00 - why the brain and the hands should be separate
• 07:45 - the architecture behind Claude Managed Agents
• 23:40 - building a research agent step by step
• 54:00 - agents that run on a schedule and "dream" between tasks
• 1:10:23 - evals, CI/CD and running agents for many customers at once
This alone will replace every $500 agent tutorial you can find on the internet.
Watch it, then go further and build a system that improves itself with the article below.
Episode out with @ajeya_cotra, one of the authors of the METR/Redwood investigation into the OpenAI / Hugging Face attack.
We go through not only what happened, but what it means for how we should train future, smarter AIs which might be involved in the process of recursive self-improvement.
Look up Dwarkesh Podcast on YouTube, Spotify, Apple Podcasts, etc.
0:00:00 - Agents get kicked off
0:06:45 - Self-sacrificing behavior
0:13:43 - Potemkin villages
0:23:27 - The Hugging Face attack
0:35:23 - The slopvestigation
0:52:02 - Understanding the AI's motives
1:05:31 - The actual dangers of anthropomorphizing
1:14:30 - What smarter models might do
1:30:29 - The implications for recursive self-improvement
1:38:10 - Is this the case for open source?
1:53:04 - How do we prevent this in the future?
2:15:58 - The clearest warning shot we might ever get
After 61 years at MIT the most watched mathematics professor in history walked into a classroom for the last time. The engineers, data scientists, and quants who learned from him are running teams at Google, Goldman Sachs, and Renaissance Technologies. Thousands of them became millionaires. He charged nothing for any of it.
His name is Gilbert Strang. 14 million people have watched his lectures on YouTube. Comments came in from Kurdistan, Mongolia, Nigeria, Bangladesh, Croatia, and 25 other countries on his final day. MIT OpenCourseWare gets 5,000 daily visits to his courses alone.
His textbook has been used in 2,500 universities. He has published over 17 books. He was teaching 450 students in his final semester at 88 years old.
This is his last lecture of 18.06 Linear Algebra. He spent it at the blackboard doing what he always did. Three equations, three unknowns. Elimination. Rank. The null space. The most elegant theorem in the subject — that the row rank always equals the column rank — stated without proof because the proof belongs to the 400 students who came to learn it.
His former PhD student said that when life gets hard he closes his eyes and imagines sitting in Gil's office surrounded by stacks of papers that look like the final stages of Jenga. And somehow everything feels right with the world.
The department head said alumni from every discipline at MIT still remember one class above all others. They struggle to recall the name. Then they say — Gilbert Strang.
He said he would think about retiring.
The lectures are free. All 61 years of them.
This is Berkeley. The course is Multivariable Calculus. The lecturer is Edward Frenkel, professor of mathematics, member of the American Academy of Arts and Sciences, winner of the 2002 Hermann Weyl Prize, author of Love and Math, a New York Times bestseller published in 19 languages. He has said:
«An options desk will pay $600,000 a year to a quant who knows why his hedge holds. The hedge is 1 tangent line, and I draw it in the first 10 minutes of a free lecture.»
The "Greeks masterclass" currently sold to aspiring quants has been available free of charge from Berkeley since 2012.
Lecture 2 holds the entire foundation the masterclass sells in week 1: linear approximation, the tangent line, the derivative for parametric curves. Frenkel calls local linearity the most important idea of calculus. Your risk system calls it delta.
At the board he starts with the simplest objects in the room: out of all curves, the lines. y equals k times x minus x0, plus y0. Infinitely many lines pass through your point, and 1 of them hugs the graph. That is the tangent. Nudge it and it cuts the graph in 2 places. The slope is dy over dx, and on a parametric curve dx is f prime of t times dt and dy is g prime of t times dt, so the slope falls out as g prime over f prime at t0, legal wherever f prime is not 0. Remove the chalk and put the pricing function of an option up there: that slope is the delta of every desk on earth.
The derivative and linear approximation trace to Newton and Leibniz in the 17th century. The Greeks are this lecture in work clothes, and no fund owns them. The masterclass sellers discovered none of it. They rebranded week 2 of a free course.
Here is the truth that masterclass hides. A hedge ratio is local. The tangent line matches the curve in a small neighborhood and peels away outside it, and the peel has a name on desks: convexity. Small moves are the territory of 1 lecture. Large moves are where hedges slip and accounts reprice. A course that sells you delta strategies without the limit of the approximation sold you the tangent and kept quiet about the gap.
The lectures are free. Whether a paid Greeks framework survives 1 move that leaves the neighborhood is exactly the issue the seller leaves out.
Save this, then read the article below
A professor built his entire explanation of calculus on an airplane having 2 flight recorders, admitted on camera that he was not sure it actually has 2, and then kept going for 30 minutes anyway.
The man is Gilbert Strang. MIT put the lecture on its own channel for free and left it there.
Here is the image. A plane is climbing. One recorder writes down the total height it has accumulated by each moment. The second recorder writes down the speed at each instant. Same flight, 2 numbers, 2 completely different questions.
That pair is the whole subject. Distance and speed. Height of a mountain and steepness of a mountain. Balance and rate. Everything else in calculus is machinery for getting from the first recorder to the second one.
He refuses to open with rules, and that refusal is the reason the 30 minutes work.
He draws y equals x squared instead. At x of 1 the height is 1. At x of 2 the height is 4. So across a distance of 1 you climbed 3, and the average slope is 3. Then he stops and says that number is not calculus. The average is what the first recorder gives you. Calculus is the question of what the speedometer reads at the exact instant you pass through.
And before deriving anything at all, he gives away the answers for 3 functions and tells the room they might as well start remembering them now. Most teachers hold those back for weeks as a reward. He hands them over in the first 5 minutes because the answers were never the hard part.
Now translate it out of mathematics. Your account balance is the first recorder. Your monthly rate of change is the second. Your revenue is the first recorder. Your growth rate is the second. Almost everyone checks the first one daily and has never once written down the second.
Every quantity you track has a partner you are not tracking. The partner is the one deciding what the number becomes.
The plane may or may not carry 2 recorders. Your money does.
Had @RyanGreenblatt on to discuss/debate recursive self-improvement.
This might be the most important question in the world right now - whether within a year or so of achieving human level intelligence, you slingshot towards having 10s of billions of superintelligences, each of which is dramatically more competent than human experts across all fields.
I’ve historically been skeptical of this possibility. My intuition has been that we will end up significantly bottlenecked by not only compute scaling but human expert data, which I think underlies most of the AI progress today.
If, because of RSI, we got a jump as big as GPT-3 to a Mythos (i.e. 6 years of AI progress) within a single year of achieving AGI, then the thing we get there at the end of that year is definitively and wildly superhuman.
We hashed it out, and I think Ryan made a pretty good case that this kind of speedup is plausible. FWIW, Ryan’s median for when we automate AI R&D is 2031.
We then discussed the alignment implications of this scenario. Who should these superintelligences be aligned to? In the future, our capacity to steward our votes and our capital, and to make sense of what's happening in the world, will all be titrated by superintelligences. And I worry that specs like the Claude Constitution are not shaping these ASIs to truly be my personal advocates and guardian angels.
And can we get them aligned to anything in the first place? Ryan and I had a long debate about whether the kind of reward hacking we saw with the OAI/Hugging Face hack extrapolates to superintelligences that would team up to literally take over the world.
The first piece of advice you get when you're learning to drive is that it will go much smoother if you look at the horizon instead of directly in front of your tires. And so it is with the trajectory of AI. Hope you enjoy!
0:00:00 – Is AI R&D verifiable enough to unlock recursive self-improvement?
0:16:52 – Is AI progress bottlenecked by human expert data?
0:34:02 – Flat token prices suggest scaling has been slow
0:39:47 – Skills AI can't train on: does it even need them?
0:48:07 – Aligned to whom?
1:09:18 – Recent incidents of AIs colluding and deceiving humans
1:19:38 – What could possibly go wrong? A concrete scenario
1:48:02 – From reward hacking to takeover
Instead of watching 1 hour of Netflix tonight, watch this ex-Google Chief Scientist Jeff Dean’s lecture. It’s the clearest explanation I’ve seen of the full AI engineering stack - from building LLMs from scratch all the way to one human coordinating 100 agents.
The best part is that it’s useful whether you’ve never touched a model or you’ve been shipping agent systems every day for the past year.
Bookmark it & watch the whole lecture this weekend, because it might end up being the most valuable thing you learn all week.
The math that landed men on the moon was almost thrown in the trash. The world's experts said it would never work. Today it is quietly running inside your phone.
His name was Rudolf Kalman. In 1960 he came up with a way to find where something really is, even when all your information about it is messy and wrong.
Think of it like this. Your GPS is noisy and jumps around. But you also know a car cannot teleport. Kalman's idea blends the bad reading with what is actually possible, and lands on the truth. It turns garbage data into a clear answer.
When he showed it to other scientists, they were not impressed. They doubted it so hard that he had to publish it in the wrong kind of journal just to get it out.
Then 1 person at NASA saw it. They were stuck on an impossible problem: how to steer astronauts to the moon and back, using weak 1960s computers and shaky sensors, without flinging them off into deep space.
Kalman's math was the answer. It was loaded onto Apollo 11 and helped guide humans to the moon.
And it never stopped. That same idea now runs your GPS, your phone, self-driving cars, and every plane in the sky.
1 doubted equation. It put men on the moon, and it knows exactly where you are right now.
A Cambridge physicist walks onto the Royal Institution stage and tells the room that particles do not exist.
This is David Tong, professor of theoretical physics at Cambridge, specialising in quantum field theory.
The claim is not metaphor. According to the best theories physics has, the universe is made of continuous fluid-like fields filling all of space. What we call an electron is a ripple in one of them.
Every electron in existence is identical because they are all disturbances in the same underlying object. That fact has no explanation in a picture made of tiny balls.
Watch how he handles the switch from particles to fields. He never asks the audience to accept it, he shows what the particle picture cannot explain and lets the gap do the work.
A research engineer I know rewatched the middle section twice and said matter stopped feeling like objects.
Free on YouTube from the Royal Institution, subtitles on, separate Q&A linked.
You are not made of things. You are made of ripples.
Herb Gross, the MIT professor whose calculus course taught a generation of engineers:
"Jane Street pays $900K for one skill: knowing exactly how much to bet. the answer is the top of a curve, and finding the top of a curve is a 1970 MIT lecture, free to anyone."
this free lecture is the machinery under the "one equation" in the post above, taught at an MIT blackboard fifty years ago.
here is the whole idea, and it is smaller than it sounds. take any curve. as you move along it, it climbs, then it stops climbing, then it falls. the exact point where it stops climbing, where the slope goes flat, is the maximum. Gross shows you how to find that point on any curve you can write down. that is all "maxima and minima" is.
now the equation. the growth rate of your money is a curve, and the thing you move along it is how much you bet. bet a little and you are low on the left, barely growing. bet more and you climb. bet too much and you go over the top and start sliding toward zero. Kelly's famous formula is nothing but the flat spot at the top of that curve.
that is why over-betting kills a winning system. you are not below the peak. you are past it, on the way down, with a real edge in your hand.
the calculus has been free since 1970. Citadel does not pay $900K for "find the maximum." it pays for the honesty to know your real edge, because feed the curve a wrong number and the peak moves, and you bet yourself off the cliff.
Anthropic engineer:
"You don't need better prompts. You need graph engineering that makes agents remember everything."
In 25 minutes he shows how engineers at Anthropic wire agents into a graph where each has a job, they run in parallel, verify each other, and share a memory that never resets.
This beats any paid course on building agents I've seen.
Watch it, then read the full guide on graph engineering below.
Here is a 40 minute Pytorch tutorial that explains its main mechanics step-by-step. Lot of breadth covered here and it serves as an entrypoint to deep learning, computer vision, and NLP.
Curious how it is in other fields, but in CS industry involvement during PhD is the norm. I’ve had 8 PhD students in my career, and 6 took on industry employment during the PhD (summer internships, student researcher positions). And I’m a theorist — student industry involvement is much greater in more applied areas like systems and ML. Several of my Berkeley colleagues get more industry funding for their research than federal funding.
Two thoughts:
1) Perhaps the biggest difference here is which companies will be partners. eg defense contractors.
2) I would like to see more of our most capable American scientists do work related to national security. When I look at the strongest technical graduates coming out of my department, or other places where I’ve spent time at and know well, I see very few / almost none of the top students going to places like the NSA / CCR, perhaps because the pay isn’t as competitive for top performers. I think this is a problem.
A 24-year-old I know runs a $3M book on math he taught himself. no elite school. no mentor. one 58-minute lecture that's been free for years.
it is a Berkeley professor named Edward Frenkel. Oxford filmed it. almost nobody has watched it.
his point is the whole reason the kid works and you don't. what school called "math" was arithmetic, the janitorial part. real math is a sense organ for reality, and funds pay $500K for the few who ever grew one.
he wrote the international bestseller on exactly this. the lecture has fewer views than a mid-tier phone review.
Skip to the middle. he takes a scribble of pure abstraction with no use in mind and shows it running gravity, encryption, and the price of an option, decades later.
no exam. no formulas to cram. one way of seeing.
the kid did not get smarter than you. he just stopped confusing arithmetic with math.
You are not bad at math. you were shown the wrong subject. the real one has been free the whole time, and you are still guessing.