Neil Movva (@neilmovva) started his career at Nvidia, working on GPUs and kernels, and has an unusually deep understanding of inference, from software to chips to power.
We spend a lot of time on each of those layers, how they connect, and where the important tradeoffs are.
What makes this conversation special is how detailed it is (like a 401-level class), yet Neil makes it remarkably clear and easy to follow.
Today he runs Sail Research, a company building infrastructure for agents to make tokens as cheap as possible.
We discuss:
- Latency versus throughput
- Why there are no bad chips, only bad pricing
- The end of kernel engineering
- Buying chips and power no one else wants
- New chip architectures
- Nvidia lore + his contrarian view of the company
- Open source and the frontier labs
I learned a ton. Enjoy!
TIMESTAMPS
0:00 Intro
0:38 Building a “Token Factory”
4:21 The Future of Background Agents
13:09 Nvidia and the GPU Stack
23:27 Chips, Memory, and Transformers
36:14 The Future of AI Training Data
44:32 Chip Scarcity and Compute Arbitrage
52:44 Reinventing the AI Data Center
59:01 Power and the “Scavenger Strategy”
1:10:10 Open vs. Closed AI
There is a process that I have used, and still use, to reignite life...
Create two timelines—6 months and 12 months—and list up to five things you dream of having (including, but not limited to, material wants: house, car, clothing, etc.), being (be a great cook, be fluent in Chinese, etc.), and doing (visiting Thailand, tracing your roots overseas, racing ostriches, etc.) in that order.
If you have difficulty identifying what you want in some categories, as most will, consider what you hate or fear in each and write down the opposite.
Do not limit yourself, and do not concern yourself with how these things will be accomplished. For now, it’s unimportant. This is an exercise in reversing repression.
Be sure not to judge or fool yourself. If you really want a Ferrari, don’t put down solving world hunger out of guilt. For some, the dream will be fame, for others fortune or prestige. All people have their vices and insecurities. If something will improve your feeling of self-worth, put it down.
Drawing a blank? In that case, consider these questions:
1) What would you do, day to day, if you had $100 million in the bank?
2) What would make you most excited to wake up in the morning to another day?
Don’t rush—think about it for a few minutes.
If still blocked, fill in the five “doing” spots with the following:
— one place to visit
— one thing to do before you die (a memory of a lifetime)
— one thing to do daily
— one thing to do weekly
— one thing you’ve always wanted to learn
What does “being” entail doing?
Convert each “being” into a “doing” to make it actionable. Identify an action that would characterize this state of being or a task that would mean you had achieved it. People find it easier to brainstorm “being” first, but this column is just a temporary holding spot for “doing” actions.
Here are a few examples:
1) Great cook —> make Christmas dinner without help
2) Fluent in Chinese —> have a five-minute conversation with a Chinese co-worker
Determine three steps for each of the dreams in just the 6-month timeline and take the first step now.
Define three steps for each dream that will get you closer to its actualization.
Set actions—simple, well-defined actions—for now, tomorrow (complete before 11 A.M.) and the day after (again completed before 11 A.M.). Once you have three steps for each of the four goals, complete the three actions in the “now” column.
Do it now. Each should be simple enough to do in five minutes or less. If not, rachet it down. If it’s the middle of the night and you can’t call someone, do something else now, such as send an e-mail, and set the call for first thing tomorrow.
If the next stage is some form of research, get in touch with someone who knows the answer instead of spending too much time in books or online, which can turn into paralysis by analysis.
The best first step, the one I recommend, is finding someone who’s done it and ask for advice on how to do the same.
@EmmanuelMacron Are you retarded? Iran’s head of state just got assassinated, 1000+ civilians dead, and their capital and other major cities are being bombed. Yet, you have the audacity to say Iran should stand down?
Your wife didn’t groom your right, you mutt