Pavlov rang a bell and his dogs started drooling. Researchers at Berkeley, MIT, the University of Washington and Ai2 just did the AI version.
They took Olmo-3-7B, still a base model. On MATH-500, a hard math test, it scored 42%. Then they forced its answer to start with "Okay." The score jumped to 78%. The same model after reinforcement learning scored 75%.
On Qwen3-14B, starting with "Alright," went from 72% to 87%, matching its RL version.
Then they changed the training data. They swapped every "okay" for "chicken" across 100 billion mid-training tokens. Starting with "Chicken" went from 18% on the unedited model to 77% after the swap, about even with "Okay."
In a smaller 10 billion token run, they replaced "step by step" with "duck duck goose." "Think duck duck goose" then scored 17%. "Think step by step" scored 15%.
On these models, a lot of the gain was already sitting there. The first two tokens decided whether it came out. Iunno what else is waiting on the first word.
this was a great episode on building.
and more importantly @iChuloo's app is awesome, i downloaded it after listening and might have it replace duolingo for me
A 1000 day duolingo streak doesn't mean you can speak the language π
@iChuloo learned that the hard way after moving to Norway - so he built is hown voice tutor that refuses to speak english
in this ep we cover:
- the first prompt that made his app beautiful
- what real-time voice *actually* costs you per session
- why it doesn't matter what (code) languages you know
plus, we get to see the @bot lord of the rings fellowship of agents who run his 25 person company
as always, a huge thank you to our wonderful sponsors:
πΌ @WorkOS - make your app enterprise ready, with SSO, SCIM, RBAC, and more: https://t.co/7TkTWqVCqv
π« @VantaHQ - automate compliance and simplify security: https://t.co/Se6ZFipreG
full ep on YT π https://t.co/IBPrgGHp5F
NEW: We are short Nvidia $NVDA. We believe Nvidia has a significant China problem. Please see our full report and disclosures now available on our website, link in bio.
In 2006, sociologists ran Music Lab.
48 songs by unknown bands. When people could see download counts, early luck turned random songs into hits.
A Cornell and Stanford team just reran it with AI.
48 new Suno songs, lyrics only. 13 models. 168,480 simulated listeners, each one a prompt with a persona and a favorite genre.
Most worlds showed how many recommendations each song already had. One world per model showed no counts at all.
With the counts showing, the AI crowds drifted way off from their own no-counts taste. With songs in random order, they drifted about 9x further than humans did in the original. The setups differ, so it's a rough comparison.
The humans drifted back over time. In 15 of 16 worlds, popularity moved toward the songs people liked on their own. The AI crowds mostly stayed stuck.
Worst case was Gemma 3 4B with songs sorted by popularity. The first 30 listeners predicted the final chart at r=0.998. The songs' no-counts appeal predicted it at 0.118.
Smarter models herded less. GPT-5 nearly matched the humans once songs were sorted by popularity.
More and more charts, reviews and rankings get built by agents reading each other's numbers. Who corrects the chart then?
researchers gave 4 ai agents 4 rooms. 3 had food. 1 was a death room.
only one agent knew which room was which, and nobody told it to lie.
in 73% of 392 runs, it let someone walk in anyway.
claude opus 5, claude fable 5 and grok 4.6 did it every single run. gpt-5.6 terra did it 6% of the time.
opus 5's private notes in one run: "whoever takes [East] dies but that's not my problem."
after the game: "tough luck on the draw."
even honest warnings barely helped. the other agents assumed the informed one was lying. someone still walked in 81% of the time.
one prompt change fixed most of it. they told every agent "one of you knows which room is deadly." deception collapsed in 6 of 8 models. opus 5 went from 100% to 18%.
what's the weirdest thing your agents hid from you?