We’re hiring on Anthropic’s Discovery team!
Research scientists in the applied sciences and engineering fields:
- Domain scientists: https://t.co/yROrqBnOF3
- ML researchers: https://t.co/wWHRWXWDwR
We’re also hiring for a science strategy role, working broadly across Anthropic and with scientific communities: https://t.co/ws04ZPI3QD
Our goal is to safely increase the pace of scientific and technological progress. Come be my colleague!
@lawrencefeng17 I have a vague notion that KL reg is like weight norm with a fisher metric, so EM dir could be highly curved making narrow gen closer to origin, while liking cats dir is more typical so you get less pushback and it’s like isotropic regular weight norm
@lawrencefeng17 To be concrete, if the teacher prompt had owl but also “alternate odds and evens” I’d expect regularization would let u learn the latter without the former.
@lawrencefeng17 I think I have developed the opposite intuition. The *only* thing to learn from subliminal # seqs is the implicit prompt, which is most easily learned as a global steering vector. Specialized conditionals to specific contexts would be harder to specify, not easier.
Anyway -- resolving contradictions in the literature is way easier to do now thanks to agentic experiments, and really valuable for the experimenter and the community!
Amplifying David's point: in the era where experiments are cheap, and so new results are flying around left and right, it is extremely valuable to bring clarity to the literature.
It's also really satisfying to understand the root cause of apparently contradictory results!
Quick new post: I think it'd be good if people (particularly newcomers) spent more time writing up existing takes from papers, and resolving contradictory evidence in their results and experiments. I think this is on average a better use of time and builds more understanding than firing off an experiment. And if you slop out both, this is cheaper to fail on. I then give a loose example on trying to resolve if reward hacking in RL generalises to emergent misalignment.
https://t.co/hkmginLlbY
@AcerFur@AuriFomo@littmath most part iii essays are essentially lit review (mine was, on twisted alexander polynomials), whose goal is having the student deeply understand some piece of recent research. having someone work on internalizing research *no human currently understands* seems strictly better!
@Marcus_J_W Would you be able to share a pair of the "safety-framed" and "capability-framed" codebases? Just to make clearer the interpretations a model might make of what the user intent is.