mholtan

[mholtan @mholtan]

@shiraeis I agree this is definitely the solution to make AI "alive". (along with 'embodiment')

[shira @shiraeis]

Found a paper that suggests we may have spent years training agents to become hunters of proxy reward when the more basic thing intelligence craves is not a reward at all, but to not run out of viable futures.

The paper proposes that behavior is best understood as maximizing future action-state path occupancy, which collapses mathematically into a discounted entropy objective. The agent doesn’t necessarily want to GET something, but rather is trying to keep as many meaningful trajectories alive as possible.

The obvious objection is “so it just does random shit? fuck around and find out?”

No, this is where it gets pretty beautiful. The agent is variable when variation is cheap and becomes surgically goal-oriented the moment an absorbing state (death, starvation, falling over, etc) gets close enough to threaten its future path space.

Variability is the same drive as goal-directedness, just operating under different constraints.

The demos are kinda wild:

- A cartpole (classic move a cart to keep a pole from falling control task) that doesn’t merely balance but dances and swings through a huge range of angles and positions because why not? The whole point is occupying state space, and rigid balance is a voluntarily impoverished life.

- A prey-predator gridworld where the mouse PLAYS with the cat, teasing it and using both clockwise and counterclockwise routes around obstacles to lure it away from the food source before slipping in to eat, using both routes roughly equally. A reward-maximizing agent would collapse to one strategy and exploit it. Here, the agent keeps its behavioral repertoire

- A quadruped trained with Soft Actor-Critic and ZERO external reward that learns to walk, jump, spin, and stabilize, and then makes a beeline for food only when its internal energy drops low enough that starvation becomes a real threat

The thing that hit me hardest is the comparison to empowerment and free energy principle agents. Both collapse to near-deterministic policies with almost no behavioral variability. This paper’s agents find the highest-empowerment state and exploit it. FEP agents converge to classical reward maximizers.

As far as I’m aware, this is the only framework that produces agents you could describe as being “alive.”

The AI implication here is that we undertrain for behavioral repertoire. Most systems hit the benchmark by collapsing onto a narrow attractor basin of good-enough trajectories. They’re competent for sure, but brittle too, with one viable plan, executed until the world shifts and leaves them with nothing.

The thing I increasingly want from agents isn’t competence per se, but option-preserving competence.

I want agents with the ability to keep multiple viable plans alive and switch between them without catastrophe.

We’ve been so focused on teaching agents what to want that we never stopped to ask what happens if wanting isn’t the point, if the deepest drive isn’t necessarily toward anything, but away from the walls closing in.

paper: https://t.co/Kn3mllmmPK

[mholtan @mholtan]

Punny Halloween Costumes: https://t.co/2dWCN38EmZ

[Iris Seraphina  @iris_seraphina]

Gotta love the elderly!! 🥹 Cute & Silly!! 🤪

[mholtan @mholtan]

@LehairJeanne Félicitations pour avoir vaincu votre "nemesis"!

[supertri @supertri_]

🔥 WHAT. A. RACE!

Jeanne Lehair wins for Podium Racing in Boston!

[mholtan @mholtan]

Still not enough talk about the problem of having a candidate who didn't win a primary. Need Ranked-Choice in all elections, so a Runner-Up is always there.

[MLFootball @MLFootball]

TRENDING: ESPN #NFL analyst Stephen A. Smith says he feels like a “fool” for voting for Kamala Harris as President.

[SpaceX @SpaceX]

Mechazilla has caught the Super Heavy booster!

[Eliezer Yudkowsky ⏹️ @ESYudkowsky]

The last years are going to be really really weird. For everyone, it's just going to happen to me earlier than to most.

[mholtan @mholtan]

@intel_beak @Aviation_Intel Those are the SpaceX StarLink satellites recently launched. It takes a while for them to separate so they look like a ribbon of lights. https://t.co/Hp1wsAgSNt

[mholtan @mholtan]

@BioEdMike Great to meet you, Michael!

[mholtan @mholtan]

@mattagon_twitch @patrickbutler @Alex79884 @elonmusk "The earth has enough resources to sustain an absolute maximum of 10 billion people" -- That's not a 'first principles' calculation. Humans only make up 0.01% of total Earth biomass, we could easily get to 0.1% if we spread out more and eat more veg. Rural areas are very sparse.

[mholtan @mholtan]

@downtohoerth DeviantArt? ArtStation?

[Cosmic Perspective @considercosmos]

Super heavy touching down on the orbital launch stand. @spacex teams ready to receive #Starship

[mholtan @mholtan]

AirBnB for fast food restaurants: AirFnF - Fast 'n Fresh?

[mholtan @mholtan]

It's been 18 months and Starship just completed an in-tact landing, but no orbit yet. Hopefully within the year.

[mholtan @mholtan]

Great webcast. 2x lifting thrust of Saturn V. 150 ton payload. 1000 cubic meters of passenger space for ~3x3x3 meter passenger cabins (100 passengers, 9m^3 per). Probably 1st orbit within 9 months, 1st Mars landing (w/o humans) within 2 years.

[SpaceX @SpaceX]

Starship at our launch facility in Cameron County, Texas. Watch as @elonmusk gives an update on the vehicle today at ~7:00 p.m. CDT → https://t.co/gtC39uBC7z

[mholtan @mholtan]

Nike VP basically embezzling money by doing this. Reminds me of Lewandowski setting up a shell company to import LIDAR/parts and forcing his employer to buy from his shell company at jacked-up prices instead of actual supplier.

[Kamala Harris @KamalaHarris]

Thank you Governor @Schwarzenegger for your powerful message to our country and for reminding us what’s at stake in this moment.

[mholtan @mholtan]

@gregkarber @lazerwalker @lbcyber Cross-platform games that work with VR *and* mainstream Desktop/Console would fix the "can't talk about with my friends" issue. WHY aren't more VR games cross-platform??? Just the asymmetry in play mechanics?? Corporate moat-building?? Frustrating!!

[mholtan @mholtan]

@godatplay Thanks for retweeting this, Josh. I'm reposting as well.