@simonw But how can you tell if it is production-quality code if you don't understand it?
I guess the challenge right now is being more productive than before while keeping control over what you ship, but it's not obvious where the sweet spot is
(Thx for answering btw, love u)
@geoffreylitt Ok but there is a point where it would have been faster if you had done all the coding yourself, keeping your understanding in the first place
@jxmnop People saying they dont write well... I do think they are way above the avg human its just that people recognize the style and start to reject it. But every writer has a style
New paper:
We train Activation Oracles: LLMs that decode their own neural activations and answer questions about them in natural language.
We find surprising generalization. For instance, our AOs uncover misaligned goals in fine-tuned models, without training to do so.
@simonw This would explain why they usually don't come up with deep or new relations, the same way an encyclopedia stores a lot of knowledge but isn't able to rearrange it. They lack the big picture
Now that school is starting for lots of folks, it's time for a new release of Speech and Language Processing! Jim and I added all sorts of material for the August 2025 release! With slides to match! Check it out here: https://t.co/pjfZsYxSk0