Kingson Man

We must know. We will know.

[Kingson Man @therealkingson]

we wrote a thing. hope you like it.

How do we design machines with something akin to feeling? (1/n)

"Homeostasis and soft robotics in the design of feeling machines" Man & Damasio 2019 Nature Machine Intelligence. https://t.co/fhM4ZbuP49
Free read link: https://t.co/VuManltFCJ

[Kingson Man @therealkingson]

"approximates the functional architecture of conscious processing... still key differences in anatomy and sense of self, lack of a body and enduring episodic memory"
–Dehaene and Naccache

🫴 "we are not thinking machines that feel, we are feeling machines that think"
-Damasio

[Anthropic @AnthropicAI]

New Anthropic research: A global workspace in language models.

Of everything happening in your brain right now, only a tiny fraction is consciously accessible—thoughts you can describe, hold in mind, and reason with.

We found a strikingly similar divide inside Claude.

[Kingson Man @therealkingson]

@RRiverosM yes! cold here but real pretty mtns!
slowly reading bolaño, quickly drinking pisco

[Kingson Man @therealkingson]

Hola from #ASSC29 in Santiago, Chile!

We are presenting early work here called "Skin in the Game": What kind of mechanism is needed for feeling-like behavior to appear in machines?

🧵...

[Kingson Man @therealkingson]

Give an agent its own agency to lose, and feeling-like signatures follow.

This poster is still early work – please share any feedback!

https://t.co/joHdNgJHOj
/🧵

[Kingson Man @therealkingson]

As an example emergent behavior, given a movable block and a den in the world, the agent runs in and seals itself off from the predator.

Bunkering down is a behavior discovered by a planner that optimized only for self-maintenance.

[SignalCreep @sucking89]

“goodbye cruel world” i say as i press the button that makes everyone nice

[The Wall Street Journal @WSJ]

The inventor behind the world-famous robot vacuum is now designing robots that form an emotional bond with their owners.

Colin Angle shared the first product of his new startup, Familiar Machines & Magic, at WSJ’s Future of Everything.

🔗 Read more: https://t.co/A1baxVNE7R

[Kingson Man @therealkingson]

https://t.co/vxFQUjQG8n

[Kingson Man @therealkingson]

"You will feel good as a pleasing completeness, a satisfying aesthetic, a sense of wholeness, almost a gravity in a certain direction."
- @kevin2kelly, A Catechism for Robots

[Kingson Man @therealkingson]

@NimaZeighami @movingsloth 🫣

[shira @shiraeis]

Found a paper that suggests we may have spent years training agents to become hunters of proxy reward when the more basic thing intelligence craves is not a reward at all, but to not run out of viable futures.

The paper proposes that behavior is best understood as maximizing future action-state path occupancy, which collapses mathematically into a discounted entropy objective. The agent doesn’t necessarily want to GET something, but rather is trying to keep as many meaningful trajectories alive as possible.

The obvious objection is “so it just does random shit? fuck around and find out?”

No, this is where it gets pretty beautiful. The agent is variable when variation is cheap and becomes surgically goal-oriented the moment an absorbing state (death, starvation, falling over, etc) gets close enough to threaten its future path space.

Variability is the same drive as goal-directedness, just operating under different constraints.

The demos are kinda wild:

- A cartpole (classic move a cart to keep a pole from falling control task) that doesn’t merely balance but dances and swings through a huge range of angles and positions because why not? The whole point is occupying state space, and rigid balance is a voluntarily impoverished life.

- A prey-predator gridworld where the mouse PLAYS with the cat, teasing it and using both clockwise and counterclockwise routes around obstacles to lure it away from the food source before slipping in to eat, using both routes roughly equally. A reward-maximizing agent would collapse to one strategy and exploit it. Here, the agent keeps its behavioral repertoire

- A quadruped trained with Soft Actor-Critic and ZERO external reward that learns to walk, jump, spin, and stabilize, and then makes a beeline for food only when its internal energy drops low enough that starvation becomes a real threat

The thing that hit me hardest is the comparison to empowerment and free energy principle agents. Both collapse to near-deterministic policies with almost no behavioral variability. This paper’s agents find the highest-empowerment state and exploit it. FEP agents converge to classical reward maximizers.

As far as I’m aware, this is the only framework that produces agents you could describe as being “alive.”

The AI implication here is that we undertrain for behavioral repertoire. Most systems hit the benchmark by collapsing onto a narrow attractor basin of good-enough trajectories. They’re competent for sure, but brittle too, with one viable plan, executed until the world shifts and leaves them with nothing.

The thing I increasingly want from agents isn’t competence per se, but option-preserving competence.

I want agents with the ability to keep multiple viable plans alive and switch between them without catastrophe.

We’ve been so focused on teaching agents what to want that we never stopped to ask what happens if wanting isn’t the point, if the deepest drive isn’t necessarily toward anything, but away from the walls closing in.

paper: https://t.co/Kn3mllmmPK

[Anthropic @AnthropicAI]

It helps to remember that Claude is a character the model is playing. Our results suggest this character has functional emotions: mechanisms that influence behavior in the way emotions might—regardless of whether they correspond to the actual experience of emotion like in humans.

[Anthropic @AnthropicAI]

New Anthropic research: Emotion concepts and their function in a large language model.

All LLMs sometimes act like they have emotions. But why? We found internal representations of emotion concepts that can drive Claude’s behavior, sometimes in surprising ways.

[Kingson Man @therealkingson]

@dodecahedra a neat but perplexing curiosity
for clarity and usability go for UpSet plot

[Kingson Man @therealkingson]

@emollick @movingsloth