Eye-opening quote from the Deepseek R1 paper on the limitations of SFT beyond just data shortage:
โFurthermore, by constraining models to replicate human thought processes, their performance is inherently capped by the human-provided exemplars, which prevents the exploration of superior, non-human-like reasoning pathwaysโ
We are in an age where human data is like gold, and the people selling the shovels (i.e. Mercor) are making a fortune.
Very interesting to imagine a future where we prove that replicating human trajectories for any subset of verifiable tasks is not actually optimal.
Pretty beautiful way to look at hiring
โA lot of companies โฆ hire people to tell them what to do. We hired people to tell us what to doโ - Steve Jobs
just heard the F15 fly over Oracle Park next to our apartment, and for a second, I thought @BainslaOmansh's face was going to be the last thing I get to peep during my short stay on Earth ๐