Looking back on the last year of shipping ML systems, I wrote a short reflection on what actually drove iteration speed.
Despite coding agents and faster V0s, the fundamentals haven’t changed much: evaluation discipline, realistic data, and tight feedback loops still matter most.
My takeaway: Speed is a consequence. Measurement is the work.
https://t.co/TkxF95Zzg0
Literacy, not a natural human state, is a learned skill that needs practice... The crucial task before us is to cultivate people’s desire to seek out cognitive complexity
from https://t.co/7ejqEg6b7s
Re: "Anyone who opens Zillow can see there are hundreds of 1-bedrooms available for less than that right now"
For a budget under $4k, there are currently 23 1 bed apts that are > 750 sq ft, and 112 1 beds that are > 500 sq ft. Only 40 of these have a dishwasher. Zillow doesn't have a filter for in-building laundry, but at least a good chunk doesn't have one. Another slice is garden/basements converted as 1 beds that are in extreme darkness
If you increase the budget to $5k, the numbers become 66 and 329 for respective sq ft.
"hundreds" is literally a few with a lot of asterisks. The city simply doesn't have enough housing with adequate amenities (or ample space for a couple)
The nature of building is changing in ways that are hard to ignore. We are all architects now. Congrats on the promotion.
New blog post:
https://t.co/4xF0Jq8db1
Activation oracles are a technique where a model is finetuned to answer natural language questions about another model's activations.
We applied them to a bunch of safety-relevant tasks and got little use out of them, and found them very hard to evaluate.
New blog post: https://t.co/HAA43fhuLa
We explore using lightweight linear probes as second-stage models to accompany LLM judges. These act as semantic filters that improve precision while maintaining high recall constraints for safety problems like adverse event detection. This was a fun research project to work on!
@fchollet The people who say SaaS is dead also seem to be the people who don't understand why companies like Salesforce exist.
It's not about the code; it's about the data that flows through these platforms and the long-term journeys of customers & their users on them.
Unfortunately, I can no longer read a "it's not this, it's that" construction without immediately raising my AI hackles. Just total rhetorical style death.
A rough example is modifying a circuit within the NN to encode desired semantics. Even though ultimately all of it goes through backprop, we can aim to constrain gradients to be aligned directionally with desired internal "structure", rather than letting semantics emerge arbitrarily.