Sanj Iyer

doing @MATSprogram | prev at @EITOxford and @LSEecon

[Sanj Iyer @sanjanaiyerr]

whipped up a short piece about why UBI and its alternatives don’t really work in a post-labour economy after ASI, my layman pitch for gradual disempowerment and what a durable system needs.

Curious to hear thoughts and feedback. Link in 🧵

[Sanj Iyer @sanjanaiyerr]

“even the futures imagined as good may preserve far less of what we love about being human than we imagine”

[Ruxandra Teslo 🧬 @RuxandraTeslo]

What are the visions of the post-AGI future in San Francisco? Many think we're all going to die, but even the "positive" visions seem rather dystopian. Here, I wrote about "handoverism," which involves handing control of the future to an AGI God, with humans outside the loop.
https://t.co/bLxgiG65vi

We are on the cusp of RSI or recursive self-improvement: this means that AI systems can self-improve, without the need for human intervention. This means our world stands to be transformed in what many call “the singularity”. I moved to SF around three weeks ago and one of the things I have been trying to understand is where we are heading, at least according to those developing the tech.

The worst scenarios are plainly bleak: all of us, or vast numbers of us, die. But what is striking to me is how little reassurance even some of the happiest scenarios offer. But one common and ostensibly optimistic vision is known as “handoverism”: humanity cedes control of its future to a godlike AI and we become the comfortably maintained inhabitants of a civilisation whose direction is no longer ours to decide. It is difficult to see what democracy would mean in such a world, or where the consent of most of humanity to the instantiation of such a regime enters the picture.

The permanent displacement of human labour is treated almost as a given. At the same time, there is scarcely any satisfying answer to the question of how we might remain useful to one another. Even our relationships seem uncertain. The promised utopia looks to me much closer to a kind of dystopia.

“It’s not my preferred outcome,” someone working at one of the big labs told me, “but once we achieve superintelligence, which we cannot stop at this point, I cannot see any other option for where humanity will go.” Then, after a pause, he added: “It is still much better than what I actually fear most, which is that we might all die.”

He sounded resigned, as if the question of what humans would be for (apparently not much?) had already been settled somewhere else and by someone else. All that remains, it seems, is to wait for this pre-destined fate to unfold before our eyes.

But I do not think that is the case. I believe we always have a choice and that we should try and steer this technology in a way that maintains our agency and control. And even if that looks hard and daunting, the immensity of the stakes here make it worth trying at least.

I do not know whether the handoverist vision will come to pass. That the people building this technology consider it plausible, and often desirable, guarantees nothing: proximity to a technology does not correlate that well with predicting its impact on society.

But they are the ones closest to it, and what they say deserves to be at least taken seriously. The rest of society, including some very intelligent people, has been far too quick to wave these scenarios away and not even get involved in the debate. Whether or not the belief is correct, it is sincerely held, and it is not “marketing hype.” I am writing this because I think it is high time that more people, inside San Francisco and out, engaged with the arguments about what our future will look like, and answered them.

[Sanj Iyer @sanjanaiyerr]

@paulmasonnews thank you

[Sanj Iyer @sanjanaiyerr]

@i_love_saes really appreciate the kind words !

[croisshauntology @croissanthology]

Post about what humans being "necessary" could mean, and how to maintain that property in the future.

[Sanj Iyer @sanjanaiyerr]

thanks Luke! our convos were super valuable for fine tuning these ideas

[Luke Drago @luke_drago_]

Great essay from our MATS fellow on why UBI isn't a durable solution to AI-driven unemployment. @sanjanaiyerr lays out four reasons:
1) No capital ownership: you get an income, but you don't own the system producing value.
2) No bargaining power: your leverage comes from what you can withhold, and if you can't without anything you can't credibly make demands. This undermines democratic systems.
3) Extractive systems adapt: governments and markets may use UBI as a license to impose more costs on assets that remain scarce.
4) People feel poor as the reference point moves: "society can provide free housing, food, transit and water, but if breathable air is scarce, an underclass still works 60-hour weeks to afford enough of it."

She also offers an analysis of common UBI-like alternatives including universal basic capital/compute, a windfall clause, and datacenter dividends.

It's ultimately unclear if any redistribution mechanism works when people lose their bargaining power. My position continues to be that we shouldn't sprint to full automation, especially without an answer to this.

[Sanj Iyer @sanjanaiyerr]

https://t.co/K2OHy4uHvo

[Sanj Iyer @sanjanaiyerr]

whipped up a short piece about why UBI and its alternatives don’t really work in a post-labour economy after ASI, my layman pitch for gradual disempowerment and what a durable system needs.

Curious to hear thoughts and feedback. Link in 🧵

[Sanj Iyer @sanjanaiyerr]

@SALTWATERSOPH thanks so much Sophie and completely agree, enjoyed your thoughtful comments

[Sanj Iyer @sanjanaiyerr]

@Anyavovaa can I read it

[Sanj Iyer @sanjanaiyerr]

@shellaneous senate house library, barbican centre, a quiet weatherspoons

[Sanj Iyer @sanjanaiyerr]

@ValsAI @croissanthology

[Sanj Iyer @sanjanaiyerr]

@dwarkesh_sp @croissanthology ur a trendsetter

[Sanj Iyer @sanjanaiyerr]

@ryanmckeen Working on research about this , can I DM you?

[Sanj Iyer @sanjanaiyerr]

@bazzlebird good luck! Econ PhD admissions I assume…did u apply to any joint / interdisciplinary programs?

[Sanj Iyer @sanjanaiyerr]

@typewriters @luke_drago_ DM’ed you!

[Sanj Iyer @sanjanaiyerr]

if you have any insights on these questions, would love to hear ! more details in Luke’s thread🙌

[Luke Drago @luke_drago_]

As AI reshapes work, which companies will capture the economic gains? Incumbent firms, AI-native startups, private equity roll-ups, or the labs themselves?

My MATS fellow @sanjanaiyerr is researching this, and she's looking for people to interview. Quick 🧵

[Sanj Iyer @sanjanaiyerr]

@sebkrier agreed it feels like it’s hard to talk about the space in between being AGI-pilled to the max and being a fixed capability over indexing skeptic

[Sanj Iyer @sanjanaiyerr]

@AaronBergman18 feels like home

[Luke Drago @luke_drago_]

Follow along as @croissanthology (one of @LRudL_ and myself's MATS fellows) explores whether there's a third way for an AI future, instead of disempowerment or augmentation.