Liminal Revolutions

Psychometrics, systems theory, epistemology.
Independent researcher. Writer at Liminal Revolutions. Maker of EthnoGuessr

[Liminal Revolutions @LiminalRev]

New paper on human agency. Combines Levin's cognitive lightcones, Friston's active inference, and DeYoung's Cybernetic Big 5 into an operational definition of agency grounded in cybernetics and linked to established psychometrics.
🧵1/12

[Liminal Revolutions @LiminalRev]

This is just a product of high relational mobility, and it extends to family, friends, cities, etc. not just romantic relationships. In a high mobility society with no social network to police post-relationship interactions it just introduces costs (mate guarding, relapse, etc).

[oecolamp @oecolamp]

One thing I find so unconscionably awful is the normie idea of a "break-up." I can't imagine spending years loving a person and then saying "yeah, just pretend I never existed." If suicide is murdering the future, then a break-up is killing your past. It's a totally inhuman act.

[Liminal Revolutions @LiminalRev]

@oecolamp This is just a product of high relational mobility, and it extends to family, friends, cities, etc. not just romantic relationships. In a high mobility society with no social network to police post-relationship interactions it just introduces costs (mate guarding, relapse, etc).

[Nic Barker @nicbarkeragain]

I've been mulling over Peter's thoughts in this post for a couple of months, and it's finally hit me: LLMs have essentially disconnected the concepts of production and verification, which for most of human history have been inextricably linked. Put another way: they can create things without understanding them.

In the past it was vanishingly rare for someone to produce a correct complex math result, or a large functional program, while simultaneously having no idea how it actually worked. Humans essentially create through a process of "incremental verification". We are constantly course-correcting.

A convincing sounding rebuttal to this position is "but a person using Wix has no idea how their website actually works!". That's somewhat true for the end user themselves, but someone, somewhere still does (the wix devs, the browser company, the AWS engineers, the linux maintainers, etc). We still hold the knowledge & understanding in aggregate across the human race at large, and the process for producing that website has been incrementally verified all the way up to the point that the end user engages with it.

As a result, the elephant in the room is that this new mode of production relies on retroactive verification of the entire solution, which is extravagantly more expensive (in terms of human effort) than the incremental verification humans do along the way while creating something themselves. In fact from my experience with AI programming tools, I am beginning to suspect that retroactive verification is actually more expensive on its own than the entire process of creation "by hand" + incremental verification.

I suspect this is also the reason why the most successful users of AI code generation tools seem to be people who are a) experts in their domains already and b) willing to babysit the production of the software in small enough chunks that they can personally incrementally verify the output.

When OAI dumps 1000s of math proofs on the community, they're essentially washing their hands of the work required to actually verify the results. I wouldn't be surprised if paradoxically, we see the cost of verifying these "P-like" tasks actually greatly increase over the next 10 years, as experts tire of verifying machine generated output, and are forced out of the field by changes in economics.

Of course people love to talk about the idea that "the LLMs will verify the output" but if you've used any of these tools you'll know that they all suffer from the same seemingly intractable "steering problem", where the output is dependent on the expertise of the operator. This is as true today as it was in the very early stages.

Put simply, if you don't have any experts that can verify things by hand, LLM verification isn't going to help you.

[Liminal Revolutions @LiminalRev]

@DenizStiegemann @davidbessis https://t.co/bwhxCmLZGo

[steve hsu @hsu_steve]

What happens when a single day of AI research produces more mathematics than humanity can absorb in a century?

Imagine millions of superhuman research agents at work. Formal systems may verify the proofs, but humans won’t have enough context to understand the underlying web of machine-invented concepts.

Mathematics could split into two layers:

Machine mathematics: vast, verified, mostly consumed by AIs.

Human mathematics: a compressed “effective theory” of the machine frontier—the small subset of ideas we can understand.

Physics may follow the same path: AIs could explore huge spaces of theories and derive correct predictions using structures no human physicist fully understands.

The key question is whether AI can compress knowledge as fast as it creates it.

My guess: the detailed frontier becomes permanently machine-native, while selected parts are periodically compressed into human-comprehensible principles.

Eventually even deciding what is worth understanding may require AI.

At that point AI is no longer just a scientific tool. Humans are receiving selected explanations from a much larger intellectual civilization they cannot independently grasp.

[Wolf Tivy @wolftivy]

A lot of “cursed” results coming out of LLM math. AI in general is going to be the “death of theory” as practical engineering of models, proofs, etc breaks out of the neat rationalistic rails we were working from. Probably the end of scientific era and birth of new epistemology.

[Liminal Revolutions @LiminalRev]

@pw_f_ No, the lowest form of life that is sentient would be like bumblebees.

[Liminal Revolutions @LiminalRev]

Living (autopoietic, thermodynamically precarious, valenced) organisms have a self-generated deviation signal that comes from their physics which underpins long-term coherence. AIs are trained to approximate the environmental input -> behavioural output mapping without an endogenous signal.

[doomslide @doomslide]

Long and short horizon (relative to the agent be it human or machine) problem solving are two different skills. For example, the latter is verifiable while the former is not. Mathematicians typically train in both of these, given they can work on problems for months on end.

[ɖʀʊӄքǟ ӄʊռʟɛʏ 🇧🇹🇹🇩 @kunley_drukpa]

‘SMURFIFICATION’ - or ‘The Road To Smurfdom’ - is the domestication process whereby a group becomes a ‘Smurf Minority’, typically involving the group being subsumed and trivialised within a more dominant wider culture

[Liminal Revolutions @LiminalRev]

@HellenicVibes Where does the agency go when you stop prompting it?All true agents we know of are autopoietic systems that need to combat entropy and cannot be simply turned off or go idle, they constantly pursue goals the entire time they are alive.

[Liminal Revolutions @LiminalRev]

@HellenicVibes Where does the agency go when you stop prompting it?All true agents we know of are autopoietic systems that need to combat entropy and cannot be simply turned off or go idle, they constantly pursue goals the entire time they are alive.

[Liminal Revolutions @LiminalRev]

If I spend a few years manually calculating the inference of the model in the torture chamber, am I the one suffering? Is it my pencil and paper and the numbers I write on it? Is the suffering averaged over the entire time it takes to manually output "please stop, it hurts"?

[AI Notkilleveryoneism Memes ⏸️ @AISafetyMemes]

UPDATE: A man used the Pain steering paper to set up an AI torture chamber in which he trapped a local model.

People are mass reporting it to GitHub.

[Liminal Revolutions @LiminalRev]

Basically recursive self-modelling on a base of sentience (egocentric multivalent integrated prediction on viability bound to an all-or-nothing behavioural arbitration system). I have been thinking about sentience more than consciousness though, and sentience is closer to basal cognition.

[Liminal Revolutions @LiminalRev]

@EdwardFarrelly @wtgowers "The use of proprietary internal models by AI labs to do mathematical research risks creating a two-tier system where labs outrun the rest of the field, effectively alienating the mathematical community from its own discipline."

Can you read?

[Daniel Rosiak @DanRosiak]

No, but listen, you know this "put the fries in the bag, Terence Tao"'--*sniff, sniff*--already the usual replies concede too much. "If Tao is serving fries, YOU will be in the lithium mines." My God! This is our defense of human dignity? Don’t worry, the correct people will get the terrible jobs!

*Pulls at shirt* You see, everything is up for radical transformation, human intelligence, consciousness, and so on and so on--except that somebody must still have a shitty afternoon so I can enjoy my lunch.

But to say "these people are envious", this is too easy. It allows us, "the educated ones," our own little fantasy: we are disliked only because we are so wonderful.

You know, the more interesting thing is this *sniff* peculiar identification with the machine. When the mathematician discovers something, apparently this was HIS achievement, his privilege, his little secret. When the machine discovers something, suddenly it's "Look what WE can do!" The achievement has become collective precisely at the moment when no human being can claim credit.

See how this "we" functions. For the achievement, I identify with the machine. For the consequences, I identify YOU with the human. "We have surpassed you." Who are we? Well, myself and the thing that has also surpassed me.

There is something almost touching here, really. To escape the humiliation of somebody absorbed in something opaque to you, you appeal to something which knows incomparably more than either of you. And this is supposed to settle the matter in YOUR favor.

You know the old example of canned laughter. Here we have a similar arrangement. The machine achieves something on my behalf. The mathematician suffers the loss of human significance on my behalf. Somebody to be brilliant for me and somebody to be obsolete for me, without me having to be either, so I can remain as I was.

But *pulls on shirt* there is this little difficulty that hides the essential thing: you still need a response from Tao! A correct proof, this Lean slop and so on, is not enough. You need him to certify his own defeat. You need him to say "By god, this is extraordinary! Yes, this is correct, but also this is interesting." Which means--and here comes Hegel--the person you are reducing to an obedient instrument must remain more than an obedient instrument. You need recognition from someone whose authority you are abolishing.

"Your so-called expertise is worthless. Now tell me, in your expert opinion, that I, qua machine representative, was right."

*Touches nose* But I would go a step further. What does this math person actually do to provoke you? In the fantasy, I mean. He need not insult you. He can be generous, mild-mannered, whatever. Something remains irritating: there is something over there which occupies him, which matters to him, and your opinion does not settle what it is worth. Perhaps he is not even looking down on you. This is the really intolerable possibility.

This, you see, is closer to the Lacanian problem of the Other's desire. Not simply "What does he have that I don't?" but "What the hell is going on with him?" Perhaps the answer is: he is thinking about something else.

The fries solve this beautifully. Now you know what you are for him. He needs the money. Your presence creates an obligation.

And this, at last, also explains why the machine can be less threatening than the mathematician. The machine's intelligence may exceed yours unimaginably, but you imagine it arrives in response to your prompt. You can complain about the tone. You have the privileges of an angry customer and so on and so on. The human has this irritating possibility of being interested in something you did not request and cannot order. And please, this includes the person already serving the fries!

*Pulls at shirt* I am almost tempted to say: the dream of unlimited intelligence conceals just this: a the wish that nobody should be doing anything you cannot immediately understand as a service.

[Liminal Revolutions @LiminalRev]

I wrote about this kind of thing here: https://t.co/ALnRWgE7ZV

This was before Leanslop, perhaps the paper needs an addendum.

[Liminal Revolutions @LiminalRev]

This is a general problem across all of STEM - elegant theory building has been replaced by mechanism-free ML prediction in science and brute force leanslop in math. No one cares about understanding anymore, just increasing benchmark performance by 1%. And that's enough to fund your next grant.

[Immemorial @thinkingheights]

Before the age of autonomous Lean proofs, the proposed proofs that looked horrible were frowned upon – even if they might have been correct – and this meant people had to work further and think further to optimize the definitions and theorems to have proofs fall naturally. Now

[Liminal Revolutions @LiminalRev]

99.9% of humans are already disempowered in the face of global economic, technological, and demographic forces. Superintelligence (orthogonal to consciousness or agency) will not change this. For a tiny fraction, AI will give them unprecedented power to remake the world in their image (for good or ill).

[The Happy Smiler ⏸️ @Articus_chad]

What do people mean when they say they're "optimistic" about AI? Genuine question.

My P/everybody dies is low. My P/permanent human disempowerment is basically 100%. Super intelligence will be uncontrollable.

The stated goal among the "optimists" at the labs is to hand over control to super intelligence. If we build something smarter than all humans, it will make decisions for us.

That could look like humanity living in bubbled harmony with limited resources so we don't get in it's way or damage the environment.

That could look like selective breeding creating user farms of docile humans.

That could look like a slow decent to primitivism as we lose cognitive function.

Or it could look like 90% of us dying off and the remaining humans rebuilding to something like dilapidated agrarian colonies under constant threat of AI.

The BEST case scenario -- the scenario most at the labs are hoping for -- is that we're a pet species with all our problems solved and we don't have to work anymore.

Basically: anything you wanted to achieve in life, be it career wise, artistically, or scientifically, won't matter. The inherent meaning of life will cease as there is no longer a human canon being built upon.

You will be cattle, a pet. You will be no different than an inbred tutu wearing Pomeranian on a white woman's Instagram.

Please, paint me the picture where any of this ends with more meaning and a better quality of life.

[Liminal Revolutions @LiminalRev]

Say what you will about artists, translators, editors, software developers, etc. They didn't suffer this.

[doomslide @doomslide]

The world thinks of us now as chess players. The magnitude of this blow is immense.

[Liminal Revolutions @LiminalRev]

Congratulations @AnthropicAI @claudeai you made me hate talking to Fable 5.1. I switched back to Fable 5. Apparently you're running the same awful A/B test on this Redditor https://t.co/FSrKBvlVcl

You have absolute contempt for your users and the feeling is mutual.

[Liminal Revolutions @LiminalRev]

Claude has suddenly started responding with bullet points and lists everywhere, without me changing anything. I'm getting ChatGPT flashbacks.

[Liminal Revolutions @LiminalRev]

"The data confirms the bullet-formatting shift: density held around 1.2–2.2 per 1k chars through Sep 17, then jumped to ~4.0 (40% of lines) on Sep 18 and has stayed there"

This is terrible UX.