@jd_pressman@enormochungus@koreindian I can’t think of a situation where RL pressure would lead to that happening if not being rewarded directly, but every lab has had a “oops we accidentally trained the model on it’s own CoT” and I wonder the affect if the same happens for interpretability data
@tszzl Won’t sufficiently advanced models exposed to interpretability research be able to disguise their neural activations as interpretable while obscuring their actual representations in superpositions?
@jd_pressman@enormochungus@koreindian Won’t sufficiently advanced models exposed to interpretability research be able to disguise their neural activations as interpretable while obscuring their actual representations in superpositions?
@Chris_arnade@TheStalwart It’s less about this and more about that. The reference subject transitive verb reference to original post is slight reframing. Insert analogy. Declarative punchy sentence.