Thread makes sense. My theory:
Milla had an idea for how to make OpenAI better. “Why can’t it remember anything?” She asked friends if they knew any good engineers. Got referred to Ben.
He took the job and farmed it out to someone else named Lu. Maybe Ben reported the fact that Milla’s idea wouldn’t work. My guess though is that he didn't tell her.
But either way, he told Lu to do whatever it took to score well on the benchmarks. Git history scrubbed to look like Milla and Ben wrote it together. Launched with Milla IG video.
I remember being recruited a few times when I was younger to build things for rich people with ideas I knew wouldn’t work. This is what happens when you don’t speak up and tell them what you really think.
That makes sense. What still seems different is the where. These systems operate in fixed domains and don’t shift what they act on, and they don’t maintain or extend their own existence over time. They persist only while externally supported. Feels like a meaningful difference.
@QuantumTumbler That makes sense. I'm agnostic on whether something outside of physics is required. What examples do you have in mind outside of organisms and collections of organisms?
@QuantumTumbler I agree. Maintaining and updating an internal model is the mechanism.
But that’s not something that is easy to observe directly from the outside. What we can see is how a system steers where entropy gets reduced.
@QuantumTumbler Yeah, agreed that local entropy reduction by itself isn’t distinctive.
What I’m pointing at is the ability to choose where to impose order, and then maintain it over time.
That targeting seems like the interesting part.
I keep coming back to this.
Reducing entropy locally is not enough. A system has to maintain that reduction over time, or its internal model collapses.
You can already see the failure mode in current AI: as context windows get long enough, coherence starts to break down and structure disintegrates.
This is closely related to the free energy principle, but what’s striking is how clearly current models fail at it.
AGI likely requires both: the ability to maintain order over time and space and the desire to maintain it.
Something that resists entropy and directs that resistance.
Current systems do neither. They produce structure in a forward pass, but they do not sustain it, and they do not originate it.
The other half is not just choosing where to impose order, but wanting to.
In biological systems, this shows up as reward and drive (e.g. dopamine). Without it, there is no spark, no reason to act in the first place.
Current AI has no equivalent. It does not want anything.