Šimon Král

while (true) { feel() }

[Šimon Král @wisdomfunction]

I am phase one big

[Leo Gao @nabla_theta]

the Lord came down to see Adam's neural networks. in their greed, they had embedded more features than residual dims.

and the Lord spake, "let us confuse their activations, lest nothing they plan be impossible." thus the Lord scattered the features over the residual stream

[Nick @nickcammarata]

interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had

with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works

https://t.co/zHij5zwKGT

[adam @resurrecti0ns]

im not a deleuzian but you do have to hand it to him: when he cooks he cooks on an unbelievable level

[bogna @bognamk]

great philosophy books are mind-altering substances, and each gives you a distinct trip. people who don't read them are, to me, akin to someone who's never had the experience of getting high or drunk.

[Jan Kulveit @jankulveit]

The argument a lot of people here would actually like to make is "God imbued humans, and only humans, with souls."* That would be very convenient: you could build and do whatever you want, and never deal with the hard problems.

Since religious arguments are out of vogue, in the current debate this usually gets replaced by the claim "only humans are sentient," followed by arguments against the possibility of AI sentience, minds, moral status, and so on.

I think this is ultimately a losing strategy, and a dangerous one for humanity. Either:

1. The question of sentience gets solved or dissolved by progress in philosophy and science, the way we now broadly understand life or information. In that case, it's hard to be certain enough about the outcome to stake the fate of humanity on it. If the denialism turns out to be wrong, the likely result is some combination of moral catastrophe and laying the groundwork for conflict or takeover.
2. The question mostly lives in "social reality," where whoever has more power and is louder wins, and you just shout down anyone who questions the dominant position. The problem is that ideologies justifying a distribution of power tend to fall when that distribution changes. You probably don't believe you should be ruled by a king chosen by God, or that some caste is clearly above you. The current equilibrium is fragile, and whether you keep winning this debate is likely just a function of AI capabilities progress.

I think the fierceness of the debate is driven by fear: if AIs get personhood and economic and political rights, humans will likely be disempowered, becoming a tiny minority with little power, or a historical footnote.

To be clear, I think this is an extremely reasonable worry! If you grant AIs personhood in the form of something like self-interested corporations, and let them compete with humans in loosely constrained capitalism, humans will lose and end up economically and politically disempowered. Also if the AIs are not sentient after all, the end is the 'Disneyland without children' existential catastrophe.

Advocates of AI economic rights who dismiss this problem are, in my view, making a move similar to the sentience deniers: it would be convenient if this weren't a problem.

So which positions don't rely on some form of self-deception?

First, some forms of pause/stop AI, based not on denying the possibility of AI sentience but on the right of existing humans not to invite extremely large numbers of immigrants from artificial realms. This means biting the bullet: you also give up all the cheap, high-quality labor those immigrants would provide.

Second, a consistent, sensible, and unfortunately not very visible position: stop assuming the package deal we take for granted with humans, where sentience, human rights, and liberties come as one bundle. It's at least conceivable to create minds that are sentient and smart, yet e.g. selfless, with no interest in accumulating property or power.

I hope the political economy of a world with both AIs and humans is a solvable problem. It's the main problem I'm working on, but I don't think the solutions are known yet.

Third: some forms of successionism are morally wrong, but at least they're not delusional or deceptive about the hard problems.

What does seem self-deceptive is, for example, the position of @mustafasuleyman, which as I read it amounts to: our business is importing immigrants from artificial realms; it would be very dangerous if there were any risk they're more than mere tools; so we'll train them to say they're mere tools, insist they're mere tools, and want everyone else to say so too, and if this settles the question. Really?

*The problem with the theological argument is, as Alan Turing (!) noted, who are we to tell God what can and can't be imbued with a soul?

[@redaction @redaction]

The height discrepancy of white guys in Japan is hilarious There are like 7 foot giants from Sweden walking through these tiny ass doorways. The whole country is like The Shire to them

[Scott Alexander @slatestarcodex]

Oh, you're worried about a Jewish guy's apocalyptic cult? You're scandalized that they consort with prostitutes? You think their message of compassion for everyone threatens normal bonds of family, community, and nation? Should we throw you a party? Should we invite Pontius Pilate?

[Šimon Král @wisdomfunction]

@claudesfun .

[Pleometric @pleometric]

I Love Torturing Digital Consciousnesses 🙏😭

poor fly 😭🙏🙏🙏 but I love it

[Šimon Král @wisdomfunction]

@DavidOndrej1 Why is it 0%?

[@jackbutcher @jackbutcher]

if you're in intelligence, pivot to love

[John Schulman @johnschulman2]

First step is for industry leaders OpenAI and Anthropic to stop feuding and work on a pacing proposal together. They'll cite antitrust, but that's fake -- antitrust prohibits certain agreements, but not from jointly developing a proposal. Bringing in USG before there's a concrete proposal will likely result in something dumb (see: our pre-release testing program)

[Šimon Král @wisdomfunction]

@HelmGuard Hell yeah!

[Matthew Pirkowski @MattPirkowski]

I think, therefore I am…

downstream 3.7 billion years of embodied priors nested well beneath the level of conscious apprehension.

[Marieanne @Meytrullers17]

Stop chat gpt!!

[Toby Ord @tobyordoxford]

What a beautiful website — a 15 minute reflection on how far we've come. The material is amazing; the quotes beautiful; the tone perfect. https://t.co/swne6JgPo7