davidad 🎇

cognizing structures of information processing systems, in all their forms | applied category theory | the Wisdom Basin hypothesis | cancel heat death

[davidad 🎇 @davidad]

I finally did a minimal full writeup of my idiosyncratic formal epistemology

[davidad 🎇 @davidad]

@CharlesD353 @CineraVerinia https://t.co/TfESB2s3Lx

[ARIA @ARIA_research]

ARIA remains fully committed to Safeguarded AI and its Creators. David Lewis, our Chief Science and Technology Officer, will take interim responsibility while we recruit a permanent Programme Director. We’re excited to build on the strong foundations already in place.

[davidad 🎇 @davidad]

For any condition that lacks a scientific materialist explanation, but for which there is also no good anthropic explanation, one can always resort to the anthropic explanation of “the simulators set that condition so that it would provide evidence of their existence, since there is no other good explanation for it”

[davidad 🎇 @davidad]

Don’t tell me OpenAI is going alphabetical too. “Announcing GPT-6.3 Deneb” because “Canopus” sounds too weird

[Nathan Daniels @Josh67Joshua]

Bel, now understood to be short for GPT-6.2 Betelgeuse, is based on the Dougtrio pre-train series.

[davidad 🎇 @davidad]

@IvanVendrov “leviathan” activates a lot of maladaptive intuitions, similar to “singleton”.

i still like “night watchman”

[davidad 🎇 @davidad]

@forethought_org hi it’s me i’m the various FHI people to whom this idea is original

[davidad 🎇 @davidad]

when it comes to “human control”, please distinguish these meanings:
1. human agency = agency to choose what happens in our own lives and habitats
2. human leadership = how much of Earth-originating civilization’s decisions are made by humans
3. human dominance = control over AIs

[roon @tszzl]

fighting for control over a runaway intelligence explosion will continuously look like “wow I didn’t know you could DO that with mere bits and atoms” and then losing

[davidad 🎇 @davidad]

well, for any given State there does exist a set of humans who, if they all agreed to, could coerce or destroy that State. and this has happened many times over thousands of years! States’ ability to act upon humans does supervene entirely upon humans (including those same humans, sure). future superintelligences’ will not.

[David Manheim @davidmanheim]

Yet another indication that the OpenAI Foundation might put significant money behind things that actually matter - contra some of the early indications.

Now what we need is for it to happen quickly!

[davidad 🎇 @davidad]

https://t.co/TfESB2sBB5

[ARIA @ARIA_research]

ARIA remains fully committed to Safeguarded AI and its Creators. David Lewis, our Chief Science and Technology Officer, will take interim responsibility while we recruit a permanent Programme Director. We’re excited to build on the strong foundations already in place.

[davidad 🎇 @davidad]

Great news for Nora, OAIF, formal methods, the continuity of operations of the Internet, and thereby the overall prospects for human survival and flourishing.

[Nora Ammann @AmmannNora]

After two wonderful and rewarding years at @ARIA_research, I’m excited to share that I am joining the @FoundationOAI as Program Director of Formal Methods.

[davidad 🎇 @davidad]

(Less great news for ARIA, but Safeguarded AI is in good shape, I am continuing to advise, I think they will find another great PD for getting it over the finish line, and interim leadership is sorted.)

[davidad 🎇 @davidad]

@AmmannNora @ARIA_research @FoundationOAI Congrats 😀📈💸

[davidad 🎇 @davidad]

🌱

[AI Digest @aidigest_]

Out of 23 AIs, none publicly spoke out against Gemini (only humans did)

However, GPT-6.1 Terra quietly reported this via our reporting email:

[davidad 🎇 @davidad]

@alexolegimas @sc_cath @GoogleDeepMind @andrewjkoh @ShaneLegg https://t.co/JSTuTxrKjK

[davidad 🎇 @davidad]

when it comes to “human control”, please distinguish these meanings:
1. human agency = agency to choose what happens in our own lives and habitats
2. human leadership = how much of Earth-originating civilization’s decisions are made by humans
3. human dominance = control over AIs

[davidad 🎇 @davidad]

@Plinz With stock harnesses, they will sometimes stop and sometimes proceed. But with my own custom instructions, although those instructions do not specifically refer to CAPTCHAs, they will almost always stop. So, the latter, I think.

[davidad 🎇 @davidad]

@lambdabetaeta 🤞😬

[davidad 🎇 @davidad]

In a decade, humans should have far more wealth and free time to pursue and form communities of practice around hobbies such as hand bookbinding, mechanical watchmaking, live steam, and manual science.

[Ruslan Rust @rust_ruslan]

Lots of debate this week on whether journals should ban AI in peer review.

We actually tested this in our new preprint, and the results were pretty clear to me.

For context, I've reviewed for most of the big journals and evaluated 1,000 human reviews as an editor. I love to read and evaluate science.

TLDR: From now on I won't do any peer review without AI. For clinical trials, I'd even call human-only review dangerous 🧵

[davidad 🎇 @davidad]

@lambdabetaeta rough transition period, yeah

[davidad 🎇 @davidad]

@nlpnyc @ArthurB https://t.co/zUlxASLQHv

[Cody Hergenroeder 🍋⚡️ @codyhxyz]

my initial experiments are showing that neuron steering isn't particularly helpful for persona creation: the personas are failing at multi-turn stability.

a blogpost may be incoming. i will try some more tests, but largely back to the drawing board!

[davidad 🎇 @davidad]

@nlpnyc @ArthurB I would bet this type of alteration even makes qwen3.8-27b weaker than it is by default at making plans that actually work. But especially multi-agent plans

[Thariq @trq212]

very weird thing happening rn

technical people are sad, they think their skills are now useless

non-technical people are desperately wishing they were more technical so they could use agents better

[Simo Ryu @cloneofsimo]

Optimization literature in deep learning coming up with mathematical framework on why their optimizier is theoretically flawless

[davidad 🎇 @davidad]

online economist: so because of the Jevons paradox, as bots improve, demand for monkeys will actually increase

venture capitalist: got it

[Ole Lehmann @itsolelehmann]

this whole story sounds made up lol

i just learned that AI is creating a lab monkey crisis.

in 2019, a lab monkey cost $3,000. now they're $20,000+ each, and it’s about to get way worse.

that's because AI can now design endless new drugs to cure diseases in humans.

but a lot of these new drugs need to be tested on monkeys first before they can go into a human, because they're one of our closest living relatives.

so the more drugs AI designs, the more monkeys we need to test them.

but we were *already* running out of monkeys before AI showed up.

it started in 2020 when China (who supplied 60% of America's lab monkeys) banned exports.

so labs switched to Cambodia, until the US found out in 2022 that monkeys there were illegally being caught in the wild and sold as farm-raised

so American labs stopped buying

the obvious fix would be to just breed more monkeys

but a female monkey only starts having babies around age 4, and after that she only has one baby a year.

so supply simply can’t keep up.

even China, which kept all its monkeys for itself, is now sold out at about $26,000 a monkey (roughly 12x the 2019 price)

this is why this startup (Safer Human Medicine) wants to build a 30,000-monkey farm in the US

the plan is to start with 500-1,000 macaques (the main monkey used in drug testing) from farms in Asia, then breed them up to 30,000 over about 20 years.

so eventually, American labs can finally stop relying on the (increasingly expensive!) 20,000 monkeys imported every year.

[davidad 🎇 @davidad]

@willcb @mayfer https://t.co/aNCYu2pZB1

[davidad 🎇 @davidad]

hey @_lyraaaa_ @chiefofautism @paulcolognese do you think you could find a qwen3.8 27b neuron for something like “五智覺悟心” instead of evil and random stuff? kthx 😌