Hopefully the FIFA disciplinary committee makes the right choice and overturns Balogun’s suspension for the Belgium game. Need him but also it’s clearly the correct call.
Becoming an RL diehard in the past year and thinking about RL for most of my waking hours inadvertently taught me an important lesson about how to live my own life.
One of the big concepts in RL is that you always want to be “on-policy”: instead of mimicking other people’s successful trajectories, you should take your own actions and learn from the reward given by the environment. Obviously imitation learning is useful to bootstrap to nonzero pass rate initially, but once you can take reasonable trajectories, we generally avoid imitation learning because the best way to leverage the model’s own strengths (which are different from humans) is to only learn from its own trajectories. A well-accepted instantiation of this is that RL is a better way to train language models to solve math word problems compared to simple supervised finetuning on human-written chains of thought.
Similarly in life, we first bootstrap ourselves via imitation learning (school), which is very reasonable. But even after I graduated school, I had a habit of studying how other people found success and trying to imitate them. Sometimes it worked, but eventually I realized that I would never surpass the full ability of someone else because they were playing to their strengths which I didn’t have. It could be anything from a researcher doing yolo runs more successfully than me because they built the codebase themselves and I didn’t, or a non-AI example would be a soccer player keeping ball possession by leveraging strength that I didn’t have.
The lesson of doing RL on policy is that beating the teacher requires walking your own path and taking risks and rewards from the environment. For example, two things I enjoy more than the average researcher are (1) reading a lot of data, and (2) doing ablations to understand the effect of individual components in a system. Once when collecting a dataset, I spent a few days reading data and giving each human annotator personalized feedback, and after that the data turned out great and I gained valuable insight into the task I was trying to solve. Earlier this year I spent a month going back and ablating each of the decisions that I previously yolo’ed while working on deep research. It was a sizable amount of time spent, but through those experiments I learned unique lessons about what type of RL works well. Not only was leaning into my own passions more fulfilling, but I now feel like I’m on a path to carving a stronger niche for myself and my research.
In short, imitation is good and you have to do it initially. But once you’re bootstrapped enough, if you want to beat the teacher you must do on-policy RL and play to your own strengths and weaknesses :)
Have you heard of the "infinite backrooms"? What about the "bliss attractor"?
I made an artifact that lets you explore the backrooms yourself. They use the new "Claudeception" tool for making artifacts that call Claude. Wanna see some examples of what you can do?
can’t forget
Remembering that you are going to die is the best way I know to avoid the trap of thinking you have something to lose. You are already naked. There is no reason not to follow your heart.
no one’s paying for a buffet of maybes. they want clarity. conviction. & a path.
a ton of builders today act like consultants… outsourcing every decision to user research or data (leave this to the big companies). it’s your job to choose without this stuff. what to simplify, what to ignore, what to say no to.
strong products come from strong taste. not consensus. & certainly not a thousand a/b tests.
omakase software
in a world where anyone can build anything, most won’t. if previous consumer behaviour tells us anything, it’s that we value convenience and and others’ taste way more than we think. even in systems designed for flexibility like notion, people buy templates
to be able to make anything is too much choice, which is its own kind of friction. most of us don’t want infinite freedom, we want a strong opinion to follow
Replit was founded in 2016.
After EIGHT YEARS of plugging away they reached $10m ARR.
SIX MONTHS after that they’re north of $100m ARR.
They added $90m ARR in six months.
That’s $493k net new ARR PER DAY.
Bananas.
(Happy investor.)
I really like the term “context engineering” over prompt engineering.
It describes the core skill better: the art of providing all the context for the task to be plausibly solvable by the LLM.