I read the DeepSeek-R1 paper the day it came out, and I don’t think GRPO is the key to its success. Instead, here’s what truly matters (ranked by importance):
1. Iterative RL and SFT
2. A hybrid reward model—mixing rule-based RM and neural RM for deterministic tasks
3. High-quality synthetic data, with human post-processing only when necessary
4. Evaluation with 64 inference samples
These open exciting opportunities for PhD students with limited compute to explore further. I might tweet some potential research projects inspired by DeepSeek-R1 later.
Beyond the technical aspects, what I appreciate even more:
1/ Openness: without it, people won’t follow.
2/ **Exceptional writing**: Strong storytelling: from proof of concept to a more complex process demonstrating full potential. Clear, easy-to-follow methodology.
Final note: Heroes admire each other, while losers resent each other. Let’s stay competitive and grateful!
Keith Rabois: “I tell founders not to worry about runway. Worry about lift.”
“If you think about lift in a plane context, a company is only valuable if you achieve lift. Runway is a tactic for achieving lift, and you may need to extend the runway so that you have more time to get lift. But unless you’re actually achieving lift with that extra time, it doesn’t help you.”
Keith continues:
“I hate when a founder is like, ‘I want to raise this much money because it gives me two years runway.’… That is a stupid way to think about your fundraising.”
Instead, founders should ask themselves what they need to achieve to achieve lift, and then work backwards from that.
When Keith invests at Khosla Ventures and Founders Fund, they write internal memos about the three key risks to the company. Usually you can’t achieve all three in one financing, so founders should be asking themselves, What’s the most important inflection? And then structure their financing to achieve that.
Keith advises founders that it’s ok to let their runway go very low if they feel like they’re approaching lift:
“A lot of founders get very bad advice like ‘Oh, you need to have this much runway or you won’t be able to raise money from strength.’ That’s nonsense. If you have traction - if you hit a viral coefficient of 1 with three months of runway - almost every VC on the planet knows how to invest in that company, and it will not be a problem.”
Video source: @khoslaventures (2024)
I was employee #30 at Facebook. Then I got fired.
For a long time, I was bitter that they let me go.
But the lessons I learned from Mark and Facebook helped me build a $100 million/year company.
10 lessons I learned from working directly under Mark Zuckerberg:
Just experimented with @vercel https://t.co/g46oQuGwvU for the upcoming landing page of my side project @beezenhq. Check out the outcome after a few tweaks!
🚀 Migrating my side project's backend to TRPC was a game-changer! Initial doubts turned into pure excitement as speed and ease took over. 💪 #TRPC#SaaS#typescript
📢 The new "flashlight measure" 🔦 command is out!
Never been easier to measure the #performance of any #AndroidDev app 🔥
(including native, @reactnative or @FlutterDev, even release builds!)
➡️ https://t.co/u1F9XRXKJe ⬅️
Here's me measuring the Twitter app on Samsung S10 👀
With @HermesEngine, JS bundle size has no impact on #ReactNative app start 🥳
- 30MB JS on the left
- 1MB on the right
Check out the full details in @El_Niebo's article:
https://t.co/3BooUipSpk
🧵 2/2