I'm Boris and I created Claude Code. Lots of people have asked how I use Claude Code, so I wanted to show off my setup a bit.
My setup might be surprisingly vanilla! Claude Code works great out of the box, so I personally don't customize it much. There is no one correct way to use Claude Code: we intentionally build it in a way that you can use it, customize it, and hack it however you like. Each person on the Claude Code team uses it very differently.
So, here goes.
We were surprised, and wanted to understand why. What about π0.5 enabled emergent human-robot transfer? We ran an experiment to test if it only appears above a certain scale.
Turns out human transfer scales with the amount & diversity of robot data in VLA pre-training!
LLMs know way too many facts. you can ask Llama 1B about the history of pizza and it gives a decent answer. it shouldn't be able to do that. it should just do a google search. use those weights for something else
The more I think about it, the more I think 2013-era morality is pretty much correct:
* free speech good
* starting companies and making good products good
* monopolies bad, vendor lock-in bad
* democracy good
* greed bad
* trying to achieve national security through oppressing people bad
* cosmopolitan humanitarian values, caring about faraway people, etc good
Things we've "learned" since then have been harmful more than helpful
We reproduced DeepSeek R1-Zero in the CountDown game, and it just works
Through RL, the 3B base LM develops self-verification and search abilities all on its own
You can experience the Ahah moment yourself for < $30
Code: https://t.co/UcGKN2SVGj
Here's what we learned 🧵
visualization of the Frisch-Waugh-Lovell theorem, which describes an intuition for where multiple regression coefficients come from.
Regression of x1 on x2 creates residuals ε = x2 - E[x2|x1]; running the regression y ~ ε yields the coefficient b2 in y = b0 + b1*x1 + b2*x1 + ε.
just as a flick-of-the-wrist pointing a laser at the night sky may cause a million-lightyear trajectory change...
...experts studying safety, compute, or policy are doomed to the whims of a tech tree that doesn't give a shit about obsoleting your life's work
Air conditioning lets you use your brain more
Students do worse when its hot. Over 13 years in NYC alone, "upwards of 510,000 exams that otherwise would have passed likely received failing grades due to hot exam conditions," and these failures delayed or stopped 90k graduations!
Easy to overlook: most people's first encounter with despotic systems is not usually with the KGB or Gestapo knocking at their door. Instead, it's usually with clean streets and improved material conditions.
I don't think the way we talk about this stuff serves us well:
People think LLMs hallucinate because of the knowledge cut-off, i.e., you are asking for new information created since training. This is true, but there is also another reason: you are asking for niche knowledge that has been "marginalized" during training. For example, when I asked GPT-3.5-turbo, "When was Jina AI founded?" I received an incorrect answer. This knowledge was definitely available before the cut-off date, but probably because we aren't that famous, it gets "marginalized" during training. This factuality issue can be easily solved with Jina Reader for search grounding, i.e., search the web, get the top 5 results, add them into the context window, and feed them to the LLMs. Seems reasonable.
But how do those new clues really contribute to the correct answer? One could say the LLM is doing some in-context learning (ICL) on those new contexts, or does the LLM "wake up" the old memory and "fix" the "flawed distribution" because of this new context?
This resonates with our recent work with @florianhoenicke on synthetic data generation for auto fine-tuning embedding models. Many people think this project is too good to be true: how can you train a good model but keep bootstrapping from a flawed distribution? How can you even generate hard-negatives? On this, I fully agree with @swyx's comment on synthetic data in one of @latentspacepod episodes: "The goal of synthetic data is less to emulate human speech; it is more to spike the distribution in useful ways." In the context of auto fine-tuning embedding models, synthetic hard-negatives do not have to be perfect; they just have to be a "wake-up call" for the model and steer it in the direction that the user points to.