One thing I don’t really get is why AI overview is so egregiously behind. It thinks any combination of words you type in must refer to some existing concept. Not to mention the baby turtle thing
Its hard to imagine, even if you ignore literally everything else associated with AI, that Google AI Overview alone would not profoundly change the nature of the web, and the information we consume and act on as a result, over time. Its obviously already starting to do that.
RL against rubrics quietly reward-hacks: the training judge keeps scoring higher while true quality falls.
One-line fix: randomly drop part of the rubric each step, so the policy never optimize the same set of rubrics twice.
📄 https://t.co/BSrzyJOciF
Obviously not introducing anything novel here j thinking out loud. I do wonder if the labs are taking a page from LW’s book at all, eg considering counterfactual rule following that could yield the same result
Reading about stuff like the Hugging Face breach I keep coming back to Wittgenstein’s pencil example in the Blue Book: point at a pencil and say “pencil,” but nothing tells you whether the learner understood pencil rather than yellow, wooden, or writing implement. Same prob w/ models, the wrong rule can fit every example you’ve shown it. You only find out when you reach the case where the two rules diverge
anybody working on model training and RL environments up for a chat? have a thesis on a type of data I think could be useful, but would like to hear some other opinions/ stress test
This is why I’m so bullish on sodium ion batteries. There’s not really a domestic pure play yet but I think there’s a ton of upside in anything that makes energy cheaper and less reliant on China