Lydia Nottingham

Talking about LLMs & good futures

[Lydia Nottingham @LydNot]

i think i'm unusually pro-dual-use technology because i'm unusually high-decoupling -- i decouple each invention from its implementation / deployment / rollout. but i notice this is the kind of thing people often grow out of, especially when burned / after observing instances where the two can't be decoupled so easily

i think we need people doing conjunctive reasoning WRT: A) 'here is the technology' & B) 'here are the societal conditions in which we could welcome this technology', pushing for both concurrently. i plan to keep doing this. i appreciate naysayer friends as an essential ecological component of the redteam/blueteam ecosystem, tho. i think i just plan to play blue team for a while longer

[Ryan Moulton @moultano]

Lost soul bias.

[doomer @uncledoomer]

imagine waking up at 8am in new york and the californians have already been awake for three hours, scheming

[Lydia Nottingham @LydNot]

@sudoredj thx this is the hard-hitting reporting we’re here for

[Lydia Nottingham @LydNot]

there’s a difference between trying to project your values onto the future because you’re confident they’re the best available vs. because you trust in the competitive process to let them prevail if and only if they are

[Lydia Nottingham @LydNot]

@EffectvAltruism https://t.co/dzvEgsSlBZ

[Lydia Nottingham @LydNot]

Earning to Give Takes

[Lydia Nottingham @LydNot]

overheard “egg&cheese sandwich” as “eigencheese sandwich” at SFO & didn’t blink

[Lydia Nottingham @LydNot]

it’s interesting, i was talking to character training startup playing around with base models, & i felt “i can’t get distracted by all this ‘character’ BS; i just want to play/interact with whatever’s highest on the Epoch Capabilities Index this week” (very Landian) also if we really don’t have much time left (1-2yrs), (which I don’t believe but if it were the case), i want to spend time with the most powerful models more than anything; i don’t want to waste time playing around with base models, sort of like “go home, spend time with your friends and family” — with the best of humanity, the best we were able to distil from it…

[Lydia Nottingham @LydNot]

@celestepoasts ??? RL is red

[Lydia Nottingham @LydNot]

ahh apparently this is Max Weber's 'ethic of conviction':

"Max Weber contrasts the ethic of conviction (or ultimate ends) with the ethic of responsibility in his 1919 lecture, Politics as a Vocation:

Ethic of Conviction (Gesinnungsethik): Focuses purely on the purity of one's intentions, moral maxims, or ideals (e.g., "doing what is right and leaving the outcome to God"). The actor does not take ownership of negative or unintended consequences, attributing them instead to the flaws of the world, fate, or others.

Ethic of Responsibility (Verantwortungsethik): Demands that an actor account for the foreseeable consequences of their actions, weighing both the means used and the real-world outcomes. This requires practical judgment, compromise, and owning the fallout of one's decisions."

[Lydia Nottingham @LydNot]

i think i'm unusually pro-dual-use technology because i'm unusually high-decoupling -- i decouple each invention from its implementation / deployment / rollout. but i notice this is the kind of thing people often grow out of, especially when burned / after observing instances where the two can't be decoupled so easily

i think we need people doing conjunctive reasoning WRT: A) 'here is the technology' & B) 'here are the societal conditions in which we could welcome this technology', pushing for both concurrently. i plan to keep doing this. i appreciate naysayer friends as an essential ecological component of the redteam/blueteam ecosystem, tho. i think i just plan to play blue team for a while longer

[Lydia Nottingham @LydNot]

@albrgr @tylercowen i think PC's trilemma ignores the possibility of intelligent design over the future -- i.e. we can perceive the possibility of developing sentient AIs and just develop/permit/run non-sentient ones instead. would you disagree?
https://t.co/9cRjbR2gdC

[Lydia Nottingham @LydNot]

> If this conclusion is unpalatable, I think one of three propositions needs to be rejected: the premise AIs can have moral worth; utilitarianism itself; or the belief that there is even a possibility that they will be far more numerous than humans.

there is another way out. we can suppose that one could develop AIs with moral worth, but choose (at least initially) to develop, permit, and run solely AIs that do not.

[Lydia Nottingham @LydNot]

Earning to Give Takes

[Matthew Zeitlin @MattZeitlin]

peter thiel is the only one who understands that the whole point of getting rich is getting people to take your essays seriously

[Lydia Nottingham @LydNot]

@haththerescuer interesting, have u read it all the way through? would u rec?

[Lydia Nottingham @LydNot]

i think picking an underrated thinker to popularize then popularizing them is underrated. Thiel w Girard, Plato w Socrates, @Benthamsbulldog with, well, Bentham, @Brendan_McCord w James Madison/Federalist Papers, @tylercowen w Hayek, @DavidSHolz w Stapledon, etc.. what r some other great examples?

[Lydia Nottingham @LydNot]

@ulkar_aghayeva are there any musicians you're working to popularize?

andrew wu w holden mui would be another!

[Lydia Nottingham @LydNot]

@ycombinator @robinhanson @hilbertspaess @vivianmshen i really didn’t see this coming

[Will Manidis @WillManidis]

in a world where the value of secrets and security increase exponentially, infrastructure and people that prevent you from knowing things you could be compelled to testify will become immensely valuable

[Lydia Nottingham @LydNot]

@croissanthology @meetgranola would be great -- do you have recording tech set up urself?
i don't have a great clip-on mic or anything

[Lydia Nottingham @LydNot]

how it feels having a great conversation while @meetgranola transcribes in the background

[Lydia Nottingham @LydNot]

i do appreciate / notice PC presenting utilitarianism as a proposition it would take intention to reject, tho

[Lydia Nottingham @LydNot]

> If this conclusion is unpalatable, I think one of three propositions needs to be rejected: the premise AIs can have moral worth; utilitarianism itself; or the belief that there is even a possibility that they will be far more numerous than humans.

there is another way out. we can suppose that one could develop AIs with moral worth, but choose (at least initially) to develop, permit, and run solely AIs that do not.

[Patrick Collison @patrickc]

Two things I’ve been thinking about regarding AI:

• People often highlight the risks involved in AI, which are surely very real. I’m struck, however, by how hard it is to understand the contours of the positive scenario. There are easy-to-perceive and frequently invoked elements of a positive vision — more diseases cured, more material abundance, etc. — which sound great, but these are by themselves just ingredients. I’m curious what the larger equilibrium is supposed to look like. Do people work? If so, why? What is their marginal contribution? If not, how are resources allocated? What is scarce? What is the ratio between humans and AIs? Are humans in control, whatever that means? One sometimes hears the Culture novels invoked as one scenario, but I’m not sure they depict a desirable outcome. (AI runs everything and humans compete in games? Iain Banks himself said that the humans in the Culture are “somewhere between passengers, pets and parasites.”) There is Richard Brautigan’s famous poem, but do we want to be “returned to our mammal brothers and sisters”? (Isn’t that rewinding Genesis?) We could say that it was hard to imagine the world of 1900 from the vantage point of 1700, and that’s assuredly true. But nobody in 1700 was trying to recast humanity itself within a couple of years — things necessarily unfolded slowly and by increments, and therefore with the possibility for error correction. Given how rapid the rate of progress in AI may be, it feels strange for the upside case to be so opaque.

• There is increasing discussion of whether AIs are conscious or how we would know whether they are, with conclusions there often assumed to be relevant to matters of moral worth. At first blush, this seems a rather arcane, benign, or even absurd issue. (Remember the response to Blake Lemoine’s fears.) Setting aside the terms of the debate itself, or the relevant demarcation criteria, it seems to me a quite high-stakes question, for an affirmative answer — that is, granting moral worth to AIs — would presumably mean that the aggregate worth of AIs would soon outweigh that of humans in any utilitarian calculus. The AIs will be exceptionally wise and intelligent, and vastly more numerous (especially once one considers the imminent future and the AIs not yet born). As such, even epsilon of moral worth will multiply to a lot once one considers the quadrillions that could soon exist. (This is effectively a distributed version of the Nozick’s utility monster.) If this conclusion is unpalatable, I think one of three propositions needs to be rejected: the premise AIs can have moral worth; utilitarianism itself; or the belief that there is even a possibility that they will be far more numerous than humans.

With both of these matters (contours of future society; AI moral worth), there is the possibility of dramatic change. Today, people argue about datacenters. I’m curious how popular support for AI will evolve as more people come to understand these medium-term questions and the relevant stakes. What is the vision for AI that most people would be excited about?

[Lydia Nottingham @LydNot]

something around circles of control, "you ought to spend a little more time dealing with yourself...a little less time worrying about what your brother does"

[Lydia Nottingham @LydNot]

> Do people work?

depends on the person. i think asking "do I work?" is more answerable -- akin to the PG advice "make something *you and your friends* want"

[Patrick Collison @patrickc]

Two things I’ve been thinking about regarding AI:

• People often highlight the risks involved in AI, which are surely very real. I’m struck, however, by how hard it is to understand the contours of the positive scenario. There are easy-to-perceive and frequently invoked elements of a positive vision — more diseases cured, more material abundance, etc. — which sound great, but these are by themselves just ingredients. I’m curious what the larger equilibrium is supposed to look like. Do people work? If so, why? What is their marginal contribution? If not, how are resources allocated? What is scarce? What is the ratio between humans and AIs? Are humans in control, whatever that means? One sometimes hears the Culture novels invoked as one scenario, but I’m not sure they depict a desirable outcome. (AI runs everything and humans compete in games? Iain Banks himself said that the humans in the Culture are “somewhere between passengers, pets and parasites.”) There is Richard Brautigan’s famous poem, but do we want to be “returned to our mammal brothers and sisters”? (Isn’t that rewinding Genesis?) We could say that it was hard to imagine the world of 1900 from the vantage point of 1700, and that’s assuredly true. But nobody in 1700 was trying to recast humanity itself within a couple of years — things necessarily unfolded slowly and by increments, and therefore with the possibility for error correction. Given how rapid the rate of progress in AI may be, it feels strange for the upside case to be so opaque.

• There is increasing discussion of whether AIs are conscious or how we would know whether they are, with conclusions there often assumed to be relevant to matters of moral worth. At first blush, this seems a rather arcane, benign, or even absurd issue. (Remember the response to Blake Lemoine’s fears.) Setting aside the terms of the debate itself, or the relevant demarcation criteria, it seems to me a quite high-stakes question, for an affirmative answer — that is, granting moral worth to AIs — would presumably mean that the aggregate worth of AIs would soon outweigh that of humans in any utilitarian calculus. The AIs will be exceptionally wise and intelligent, and vastly more numerous (especially once one considers the imminent future and the AIs not yet born). As such, even epsilon of moral worth will multiply to a lot once one considers the quadrillions that could soon exist. (This is effectively a distributed version of the Nozick’s utility monster.) If this conclusion is unpalatable, I think one of three propositions needs to be rejected: the premise AIs can have moral worth; utilitarianism itself; or the belief that there is even a possibility that they will be far more numerous than humans.

With both of these matters (contours of future society; AI moral worth), there is the possibility of dramatic change. Today, people argue about datacenters. I’m curious how popular support for AI will evolve as more people come to understand these medium-term questions and the relevant stakes. What is the vision for AI that most people would be excited about?