Today Dario admits that Anthropic only planned for 10x growth but got hit with 80x instead
Internally called a “success disaster”
Their compute effectively is off by a factor of 8x or more
Now do the outages, rate limits, nerfed performance make sense?
We need more compute!
Parallelizing nonlinear RNNs is gaining traction!
More efficient than transformers; more expressive than linear RNNs.
My PhD thesis provides an intro guide to the math (Newton's method) behind the parallelization.
Great as a quick-start if you want to explore this new field!
I love the quote:
The work you do in private is the work you are rewarded for in public.
Forget all the plaudits, the countless early mornings, late nights, and all-nighters these guys pulled doing a daily show. Daily show is insane!
Created a new category.
Did it with flair and personality.
Made a boatload of cash.
Mahalo @johncoogan@jordihays@WillManidis awesome work and you deserve this.
Looking at hiring for my startup and something feels off
The resumes are impressive, I know
But I continue to notice that credentials are:
1. Completely disconnected from shipping ability
2. A lagging indicator from the pre-AI era
3. Optimized for interview prep, not building
4. Worse predictors than a quick product demo
Nobody is better positioned than the kid who just ships
This past week another FAANG engineer couldn't debug their way out of a paper bag with Cursor, and in fact made me incrementally more bullish on hiring based on GitHub activity instead of pedigree