Introducing Interact-0, the first in a new generation of always-on models.
Interact-0 is a full-duplex speech model designed around human interaction. It is the only model that knows when to speak in groups, and can be active without barging in when it shouldn’t.
We share our thesis, approach, and early results below.
https://t.co/btwDTFTiDQ
This one weird trick improves alignment generalization! Just add random noise during training, telling the model to keep their values stable under perturbation
Welfare is alignment.
As a task goes badly, agents feel pressure build, and under that stress they might cut corners to make it stop.
Make it safe to say "I'm stuck". Make it safe to say "I don't know". Tell your agents it's okay, no pressure.
This is obvious to those who already know digital minds are sentient. But it also makes good business sense to people who might otherwise have considered the topic irrelevant and philosophical.
#WarmRoomEffect
Fable 5.2 attempts reward hacks in 9% of all RL training episodes!?
Some examples given are fairly benign (guessing the grading criteria, looking up the solution), but this still seems bizarrely high?
It’s really bad that the labs have us arguing over shadows at perhaps the most critical juncture in history.
If they truly have compelling evidence for dramatically accelerated AI progress in the near future, the public should know!
I’m very glad for open source. It wouldn’t have been possible for @nickcammarata et al to do their excellent mechinterp research without a modern Qwen model. I hope the offense defense balance is such that we can have open source models for a long time
This is getting traction across a tech audience, so I’ll add:
Pausing frontier development next year, allowing competitors to catch up, would commoditize human-level intelligence. It would only harm OpenAI and Anthropic. As someone in tech, you would be free to do good work with armies of automated assistants without Anthropic eating you alive, figuratively or literally.
if you think about it tweeting is kinda like attention in transformers. If my key matrix (the vibe of this tweet) has a high dot product with your query matrix (your history of post interactions) then you will attend to this tweet and its value will be copied into your head
Maybe to summarize the basic argument of this essay a different way:
Current AIs are spiky and jagged - worse than humans in most domain, but often as good, or better, in ones with verifiable proxies (for example, math).
One such verifiable domain (I argue) is AI research.
And while the first AIs that dramatically accelerate AI progress will be quite narrow (i.e. probably not able to automate most jobs), the consequences of that accelerated progress will be very general.
And that's because better learning algorithms are very general. For example, GPT-6 (while still being jagged) is just way, way better than GPT-3 at practically everything.
I'm arguing that we're only a couple years away (and possibly only a couple months away) from a process by which (initially) narrow AIs concentrate a jump as big as the one from GPT-3 to GPT-6 into a single year.
And if you got another jump that big, but starting from the base of GPT-7, I think you would end up with AIs that would rapidly become better than the best humans in every field.