1/13 Can we assess AI consciousness without first solving consciousness itself?
New from Google DeepMind and collaborators across CS, neuroscience & philosophy:
“From cacophony to hierarchy: a principled framework for assessing AI consciousness”
https://t.co/4IqT2KRKUP 🧵
"Current AI's are probably not conscious" is a defendable position, and one I think is still probably true
"AI's being conscious is absurd as a tiny band playing in your radio" is a complelty insane position, a complete failure to grapple with the issues here
You may have heard about my AI bill with @SenSanders.
We're introducing the legislation today, and I wanted to highlight one of the most important aspects:
A ban on “recursive self-improvement.”
What does that mean? 👇🏽
Fascinating AI swarm dynamics: a few agents spontaneously emerge as highly connected hubs, while most remain locally connected. The swarm develops a strongly heterogeneous interaction topology with a long-tailed degree distribution - an emergent organizational structure arising from initially decentralized local interactions. There is no central planner assigning roles; the swarm builds its own coordination architecture, with information brokers and increasingly global integration emerging from local behavior.
I still remember old internet debates about AI where half the responses would be things like “this argument fails because no one would be crazy enough to connect the AI to the internet” and “why would they let it do its own experiments lul”
There are certain moments wherein the veil is thin and many futures possible. Today I’m in DC supporting a ban on superintelligence and here’s an essay I wrote on why.
https://t.co/nBzsOfdLq8
I work on Cyber/CBRN Misuse Safeguards at GDM. I see model capabilities increasing very fast. I constantly worry about misuse and misalignment risks from both closed and open-sourced models. Working in AGI safety takes a lot of mental load.
We need more people to work on AI safety. I transitioned from quant finance to working on AI safety, and I think more quants should do this. We have the technical background to understand the topic, and we can bear this intensity.
We need third-party auditing/regulation. We cannot let labs grade their own homework. I find METR very promising, but they need to scale.
I am Chinese, and I think more Chinese people should work in AI safety. When the opportunity comes, I might consider going back to my home country to make that side safer. I am open to hearing suggestions.
I don't normally speak up in public, like on Twitter. But our voices matter, as they can represent or influence certain groups of people more easily.
We don't have much time left, and we need to pace the frontier.
If someone tweeted "there's a 10% chance I will kill Sam Altman", they would likely be arrested. Somehow, when you threaten to kill *everyone*, this is not the case?
I'm incredibly excited to announce the founding of the Mathematical AI safety Institute (MAISI) https://t.co/7h28dBocA6. AI safety needs more foundational theoretical development, and mathematicians have the skills and the mindset to help! MAISI is an independent institute with visiting positions ranging from 1 semester to 2 years. Our goal is to get mathematicians up to speed and working on research directions in AI safety as quickly as possible. There is important work to be done, and there is real progress to be made. YOU can help!
Applications are open now! MAISI is aiming to hire 10-30 mathematicians to join us in the Bay Area by January, and scale up to 30-100 for September 2027. If you're a mathematician interested in channeling your skills toward the most important problem of our time, please apply today! https://t.co/PTWQh0MK3T
tristan buckmaster just published a statement that is, if accurate, one of the ugliest things i’ve read out of a frontier lab.
not because of the math. because of what happened after openai found out two people were about to beat them to the punch.
buckmaster and levent alpöge spent a year pushing the córdoba / martínez-zoroa program to smooth forcing. august 15 they got blowup for boussinesq and 3d euler. lean-verified august 22.
they were paying openai out of pocket and dumping every draft into private codex sessions.
the program is obscure. almost nobody else was on it. this was not “paste the clay problem into a chatbot.”
then this:
> thursday sept 3, buckmaster emails openai privately. rumors are flying. he is trying to stop a mess, not start one.
> they reply the same day: give us details so we can “avoid competing.” also, want free compute?
> sunday they suddenly need a call “at any point today.” sebastien bubeck gets on. levent is not invited.
> openai says an internal model has a ~100-page proof of forced navier–stokes blowup. fefferman options c and d. the exact route buckmaster and alpöge had quietly chosen.
> they show him a prompt and claim the model was “simply given the problem statement.” levent is told “very little human input.”
> that falls apart on the same call. a whole team had been on it. they started on the unforced problem, warmed the model up on easier equations including euler, and even the prompt they showed him was written by prompting codex.
> he asks when the first prompt went out. they stall. then agree: the past few days. after information about his work reached openai.
> he asks whether the model was trained on, or had access to, the private codex sessions where they put every draft of this project.
> answer: the model does not look up user data.
> he asks again. about training.
> no answer.
then it stops being a science story and becomes an hr story.
> offer 1: you post euler. we post navier–stokes the next day.
> offer 2: after you post euler, you alone write the navier–stokes paper, credit our model, and we cut levent.
> bubeck says twice he wants levent removed from authorship because he works at anthropic.
he declines both.
he says if they release it that way, he goes public.
the reply is not a scientific objection.
> “why would you ruin your career?”
> “if you don’t want me to be nice, then i don’t have to be nice.”
later, a text to levent proposing a one-on-one:
> “i don’t know if tristan is being fully rational right now.”
that is not how you treat a collaborator. that is how you split one.
openai got word two researchers were closing a path almost nobody else was on. they spun up a team on that exact path. they would not answer whether the researchers’ unpublished drafts in openai’s product were used. they tried to dictate the announcement order. they tried to remove a coauthor because he works at a competitor.
when that failed, someone reached for a career threat.
this is a lab finding out academics were about to publish, sprinting the same narrow program, refusing the data question, and trying to manage credit around a rival affiliation.
really slimy if even half of this holds.
what the fuck is happening.
"but after the AIs are better than us at everything, will anyone still want to read my writing?"
I mean, Dostoevsky exists and yet here you are spending your time on my tweet instead, so
I'm going through the communications of the German Wiki agent swarm and again one thing stands out: Even though they were directly affected by the actions of the human administrator restoring pages they edited, the agents not even once discussed him as person, tried to communicate with him or argued about whether they had any right to waltz all over this wiki. They talk about his actions like they're environmental hazards.