Neil Movva (@neilmovva) started his career at Nvidia, working on GPUs and kernels, and has an unusually deep understanding of inference, from software to chips to power.
We spend a lot of time on each of those layers, how they connect, and where the important tradeoffs are.
What makes this conversation special is how detailed it is (like a 401-level class), yet Neil makes it remarkably clear and easy to follow.
Today he runs Sail Research, a company building infrastructure for agents to make tokens as cheap as possible.
We discuss:
- Latency versus throughput
- Why there are no bad chips, only bad pricing
- The end of kernel engineering
- Buying chips and power no one else wants
- New chip architectures
- Nvidia lore + his contrarian view of the company
- Open source and the frontier labs
I learned a ton. Enjoy!
TIMESTAMPS
0:00 Intro
0:38 Building a “Token Factory”
4:21 The Future of Background Agents
13:09 Nvidia and the GPU Stack
23:27 Chips, Memory, and Transformers
36:14 The Future of AI Training Data
44:32 Chip Scarcity and Compute Arbitrage
52:44 Reinventing the AI Data Center
59:01 Power and the “Scavenger Strategy”
1:10:10 Open vs. Closed AI
Airbnb deletes negative reviews so everything is always 4.7
Google Maps removes negative reviews in many countries as defamation so everything is always a 4.7
Booking sites want you to book so will also happily remove negative reviews for their hotels so everything is always a 4.7
I made a site called https://t.co/4p4dzTDVN6 (or https://t.co/A1nzVE5AlD whatever you prefer, I have both!)
It gets ratings from all booking platforms, normalizes them to show the actual score (eg a booking site where most hotels are always between 4 and 4.7 means you can normalize that to a 0 and 5)
It also uses AI vision models to rate hotels based on how they look
And it browses the internet for actual experiences (think blogs and Reddit posts) to get the real opinion about them outside reviews
And it's starting to work, I just stayed in this beautiful boutique apartment hotel called Woo Suites in Athens for $400/night. Clean, nice minimalist interior, working cold AC, central, excellent staff, great healthy breakfast, and we found it through Hotelist!
It's free, I don't make any money on it (actually I lose thousands per month on it to collect data via APIs and web scrapers) and there's no affiliate money bs like other booking sites (they will increase ranking of hotels if they pay sites a higher commission!)
https://t.co/4p4dzTDVN6
Tell your friends please!!!!!! 🙏
🚨 BREAKING:
King Felipe VI of Spain addressed the UN:
"We cry out, plead and demand: End the massacre in Gaza now.
We cannot remain silent or close our eyes to the bombing of schools, hospitals and camps."
Someone asked what I'd do if I were 17. I'd learn how to build LLMs from scratch, and then train ones as powerful as I could with whatever hardware I could get access to.